RAG pipeline that embeds your charts instead of ignoring them

What it does

NexusRAG ingests PDFs, Word files, and PowerPoints through either Docling or Marker, then answers questions with inline citations tied to specific pages and headings. It combines ChromaDB vector search, a LightRAG knowledge graph, and cross-encoder reranking to surface context before handing off to either Gemini or a local Ollama model. Images and tables are captioned by a vision LLM and folded into the text chunks they share a page with, so searching for a chart does not require a separate image index.

The interesting bit

The pipeline assigns two different embedding models with different jobs: a fast local bge-m3 for vector retrieval, and a separate, richer model for knowledge-graph entity extraction. During retrieval, vector over-fetch and graph lookup run in parallel, then a cross-encoder rescores the top twenty candidates jointly with the query. It is a rare case where the architecture diagram actually matches the code.

Key highlights

• Dual-parser support: switch between Docling and Marker via environment config without touching downstream logic.
• Visual content is embedded through captions appended to page chunks, making images and tables searchable through standard vector similarity.
• LightRAG runs on file-based storage (NetworkX plus NanoVectorDB), so the knowledge graph needs no extra Docker services.
• Citations use four-character IDs with page numbers and heading paths, and the UI lets you click through to the exact source section.
• LLM provider toggles between cloud Gemini and local Ollama via environment config; the system auto-detects native tool-calling support versus prompt-based fallback.

Caveats

• The README warns that smaller local models (around 4B parameters) may fail to extract any entities from complex documents, so knowledge-graph quality depends heavily on model size.
• Docling’s formula enrichment can consume roughly 18–20 GB of VRAM, while Marker sits closer to 2–4 GB.
• Gemma 4 models require Ollama v0.20.0 or newer.

Verdict

Worth a look if you need a batteries-included RAG stack that preserves document structure and cites sources out of the box. Skip it if you want a minimal, pluggable component; this is a full-stack opinionated assembly.