Seven weeks of RAG that treats keyword search as a feature, not a bug

What it does

This repository is a seven-week, notebook-driven course for building a production-grade research assistant called the arXiv Paper Curator. Learners progress from Docker-based infrastructure and Airflow ingestion pipelines to hybrid search, local LLM integration, and finally agentic RAG with LangGraph and a Telegram bot. The stack is deliberately grounded in OpenSearch, FastAPI, PostgreSQL, and Ollama rather than managed black-box services.

The interesting bit

The curriculum inverts the typical tutorial sequence: it forces you to master BM25 keyword search and relevance scoring in OpenSearch before ever adding vector embeddings. That “search first, vectors second” philosophy is the core thesis, and the Week 7 agentic layer—complete with query rewriting, document grading, and out-of-domain guardrails—acts as a capstone rather than a shortcut.

Key highlights

• Seven staged releases (week1.0 through week7.0) covering infrastructure, ingestion, BM25, hybrid search, RAG, monitoring, and agentic orchestration
• Uses OpenSearch 2.19 for hybrid retrieval (BM25 + semantic), not a pure vector database
• Local-first stack: Ollama for LLMs, Gradio for UI, and Langfuse for tracing without mandatory cloud inference
• Week 7 adds LangGraph decision nodes, adaptive retrieval, and a Telegram bot interface
• Each week pairs a Jupyter notebook with a Substack blog post for context

Caveats

• Requires significant local resources: 8GB+ RAM and 20GB+ disk space for the full Docker Compose stack
• Needs external API keys for Jina embeddings and Langfuse to function fully; the .env defaults alone are not sufficient
• The repository is explicitly a learning scaffold, so production hardening beyond monitoring and caching is left as an exercise

Verdict

Ideal for backend engineers and ML-curious developers who want to understand why retrieval fails in production, not just how to call an embedding API. Skip it if you are looking for a drop-in SaaS component or a pure theory course without wiring services together.