marella/chatdocs
Offline document Q&A tool that runs local LLMs (Transformers, GGUF, GPTQ) with a web UI to chat over PDFs, Word docs, and other file types.

ChatDocs is a local RAG (Retrieval-Augmented Generation) system that embeds documents into a vector store (Chroma) and answers questions by retrieving relevant chunks and feeding them to a local LLM. It supports GGUF/GGML models via CTransformers, Hugging Face Transformers models, and GPTQ quantization. A YAML config controls embedding models, chunk sizes, retrieval parameters, and LLM options. All processing happens on-device with no data leaving the system.