amazon-science/RAGChecker
A fine-grained evaluation framework for diagnosing RAG system performance using claim-level entailment metrics.

Velocity · 7d
+1.5
★ / day
Trend
→steady
star history
RAGChecker is an automatic evaluation framework designed to assess and diagnose Retrieval-Augmented Generation systems. It provides comprehensive metrics including overall system assessment, diagnostic retriever metrics, and diagnostic generator metrics. The framework utilizes claim-level entailment operations for fine-grained evaluation and includes a benchmark dataset with thousands of questions across multiple domains.