A Top-3 RAG Stack That Ditches Fine-Tuning

What it does

This is a retrieval-augmented generation pipeline tailored for automated network operations. It answers domain-specific questions by chaining custom document chunking, dual-route sparse retrieval, BGE-based reranking, and LLM generation. The system took third place in the CCF AIOps 2024 Challenge and placed first in the GLM4 preliminary track.

The interesting bit

The framework deliberately avoids fine-tuning any models, relying instead on BM25, dense retrieval fusion with RRF, and off-the-shelf rerankers to stay accurate. It also treats inference acceleration as a set of plug-and-play modules you can drop into coarse ranking, reranking, or generation without rebuilding the pipeline.

Key highlights

• Competition results: 1st in the GLM4 preliminary round, 2nd in the semifinals, and Top 3 overall
• Runs on a single GPU with 16 GB of VRAM and requires no model fine-tuning
• Combines Chinese BM25, Qdrant dense retrieval, and RRF fusion for coarse ranking
• Includes an OCR module (paddleocr+glm4v) for extracting answers from images
• Ships with FastAPI and Streamlit frontends

Caveats

• The README explicitly warns that only semi-final code is documented, so large parts of the codebase are unexplained.
• It is built around GLM API keys and Hugging Face-adapted LLM wrappers intended for use in China.
• A 16 GB GPU is the stated floor, which may still exclude lightweight CPU-only deployments.

Verdict

Worth a look if you need a battle-tested, domain-specific RAG baseline that stays lean. Skip it if you require fully open LLM endpoints, exhaustive documentation, or a pure-CPU stack.