The Multi-Query RAG Repo That Proves Hybrid Search Wins

What it does

The repo implements a retrieval pipeline that asks an LLM to rewrite a user query into several variations, runs each against ChromaDB (and optionally BM25), then merges the ranked lists with Reciprocal Rank Fusion. It ships with a full evaluation harness built on the NFCorpus medical dataset from BEIR, computing precision, recall, NDCG, and MRR with paired-bootstrap confidence intervals.

The interesting bit

Most repos would ship a toy demo and call it a day; this one includes a replication of an arXiv paper, an end-to-end LLM-judge answer-evaluation driver, and the frank admission that vector-only RAG-Fusion collapses toward zero benefit once a cross-encoder reranker enters the picture. The hybrid variant — BM25 plus vector search multiplied across diverse LLM rewrites — is where the statistically significant gains live.

Key highlights

• hybrid_diverse+rerank produces measurable lifts in both retrieval ranking and generated answer quality over baseline retrieval at n=200 with confidence intervals.
• The evaluation suite covers retrieval-only metrics, cross-encoder reranking, end-to-end LLM-judge answer scoring, and cost/latency analysis across corpus sizes.
• A “free lunch” insight: fusing BM25 and vector results via RRF costs nothing extra yet improves ranking quality, especially at the top of the ranking.
• The diverse prompt strategy — explicitly requesting synonyms and varied specificity — modestly outperforms standard query rewriting.
• Includes replication materials for arXiv paper 2603.02153v1 under experiments/arxiv-2603-02153-replication/.

Caveats

• Retrieval-only results overstate the benefit; with cross-encoder reranking and end-to-end LLM-judge answer evaluation, lifts shrink though they remain statistically significant for the hybrid variant.
• Vector-only fusion variants are roughly a wash on average and net-negative on rich queries once a strong reranker is added.
• The README recommends adaptive routing in production to avoid running fusion on every query, since easy queries can regress and LLM rewrites add cost and latency.

Verdict

Worth studying if you are building retrieval pipelines for medical, legal, or scientific corpora where terminology mismatch is common and recall trumps precision. Skip it if you need sub-second p95 latency, run FAQ chatbots, or believe that vector-only query rewriting is a magic bullet — the numbers here say it is not.