A control-freak RAG agent that actually reads the manual

What it does

This project implements a Retrieval-Augmented Generation agent that answers involved questions from your own documents using a deterministic graph architecture. It preprocesses PDFs into chunks, chapter summaries, and a quotes database, then executes multi-step plans to retrieve and reason over the content. The agent evaluates its own output with Ragas metrics for correctness, faithfulness, and relevancy.

The interesting bit

Before planning, the agent anonymizes questions by replacing named entities with variables, which forces the LLM to sketch a generic reasoning path without leaning on its pre-trained knowledge. It then de-anonymizes the plan into concrete retrieval or answering tasks, verifies that generated content is grounded in the original context, and re-plans remaining steps based on what it finds.

Key highlights

• Deterministic graph-based “brain” that breaks complex queries into manageable sub-tasks and adapts the plan as new information arrives
• Question anonymization to prevent the LLM from exploiting pre-trained knowledge instead of retrieved documents
• Multi-layered retrieval across chunked content, chapter summaries, and specific book quotes
• Hallucination checks that verify answers are grounded in source context before final generation
• Built-in Ragas evaluation for answer correctness, faithfulness, relevancy, recall, and similarity

Verdict

Worth exploring if you are researching multi-step RAG architectures or need a reference implementation for controllable document Q&A. Skip it if you are looking for a drop-in library or a production-ready backend service.