huggingface/lighteval
A benchmarking and evaluation framework from Hugging Face for assessing LLM performance on standard benchmarks.
Not currently ranked — collecting fresh signals.
star history
Lighteval is a comprehensive evaluation toolkit designed by Hugging Face’s Evals Team to benchmark LLMs across diverse backends. It enables standardized performance measurement using existing tasks and metrics, with support for custom evaluation scenarios. Results are saved with detailed, sample-level granularity to support debugging and comparative analysis across model runs.
Frequently asked
- What is huggingface/lighteval?
- A benchmarking and evaluation framework from Hugging Face for assessing LLM performance on standard benchmarks.
- Is lighteval open source?
- Yes — huggingface/lighteval is open source, released under the MIT license.
- What language is lighteval written in?
- huggingface/lighteval is primarily written in Python.
- How popular is lighteval?
- huggingface/lighteval has 2.5k stars on GitHub.
- Where can I find lighteval?
- huggingface/lighteval is on GitHub at https://github.com/huggingface/lighteval.