Query Piles of PDFs with a Single Adaptive RAG Pipeline

What it does

This is a retrieval-augmented generation setup that lets you ask questions across multiple PDF and TXT files in a single conversation. It feeds retrieved context from your documents to various backend LLMs—OpenAI GPT, Anthropic Claude, or local quantized models like Llama 2—through a terminal interface. The core pitch is that you are not restricted to one document at a time or to rigid text splits.

The interesting bit

Instead of chopping documents into fixed-size blocks, it uses a sliding-window chunking mechanism that dynamically adjusts window size and position based on data complexity. This is paired with an ensemble retriever that attempts to balance semantic similarity against precise keyword-style matching, aiming to solve the usual trade-off between context breadth and retrieval accuracy.

Key highlights

• Multi-document QA: handles simple and multi-hop queries across several PDFs or TXT files at once, rather than locking you into a single document.
• Adaptive chunking: a sliding-window approach tries to preserve both fine-grained details and coarse-grained context instead of relying on static chunk sizes.
• Broad LLM support: works with OpenAI GPT, Anthropic Claude, and local open-source models (notably Llama 2, including GGUF quantized versions).
• Hardware honesty: the README notes you’ll need more than 35 GB of GPU RAM to run the recommended local GGUF model, and suggests Together.ai as a lighter alternative.
• Built on familiar pieces: leans on LangChain, ChromaDB, and llama-cpp-python for retrieval and local inference.

Caveats

• Citations are not yet implemented, so the tool won’t show you exactly which chunk or page produced a given answer.
• Asynchronous processing is limited, which may bottleneck large ingestion or query workloads.
• Only PDF and TXT formats are supported for now; OCR and additional file types are listed as upcoming.

Verdict

Worth a look if you need a terminal-based RAG experiment that spans multiple documents and you want to compare commercial and local LLMs without rewriting chunking logic yourself. Skip it if you need a web UI, source citations, or production-grade async handling today.