A local chatbot that threw the whole RAG stack at your PDFs

What it does This chatbot is a Streamlit app that ingests PDFs, DOCX, and TXT files and answers questions using a pile of local retrieval techniques. It runs entirely offline through Ollama, combining dense vector search, sparse keyword matching, knowledge graphs, and cross-encoder reranking to surface relevant chunks. The app also maintains chat history so follow-up questions retain conversational context.

The interesting bit The project treats RAG like a buffet rather than a menu: it layers BM25, FAISS, GraphRAG, HyDE query expansion, and a cross-encoder reranker on top of local Ollama inference. That kitchen-sink approach is unusual for a single offline demo, though the README leaves unclear how tightly these pieces are integrated versus simply being called in sequence.

Key highlights

• Runs fully offline via Ollama; no API keys, cloud uploads, or subscriptions required.
• Hybrid retrieval pairs BM25 sparse search with FAISS dense vectors, then adds GraphRAG for relational document context.
• Neural reranking (cross-encoder) and HyDE query expansion attempt to boost relevance before generation.
• Chat history integration preserves context across multi-turn conversations.
• Ships as a Streamlit app with Docker support for containerized deployment.

Caveats

• The README claims DeepSeek-7B generation, but the install script and Docker config default to a huihui-ai/Qwen3-1.7B-abliterated model—so the actual out-of-the-box LLM is ambiguous.
• Several touted capabilities—model selection UI, suggested questions, and alternative vector stores like Pinecone or PostgreSQL—are listed under “Upcoming Features,” not implemented.
• The project appears to be a high-level orchestration of existing local-AI tools rather than a novel retrieval engine.

Verdict Worth a spin if you want a ready-made, fully-local interface to interrogate documents without OpenAI bills. Skip it if you need production observability, rigorous evaluation, or a clean abstraction over the underlying model stack.