EvolvingLMMs-Lab/lmms-eval
Unified evaluation toolkit for benchmarking multimodal large language models and vision-language models across diverse task types.
Not currently ranked — collecting fresh signals.
star history
LMMs-Eval is a comprehensive benchmarking framework for evaluating multimodal AI models including VLMs and LLMs. It provides standardized benchmarks across text, image, video, and audio modalities with support for over 100 evaluation tasks and 30+ models. The toolkit is designed to systematically probe and measure model capabilities in real-world scenarios.
Frequently asked
- What is EvolvingLMMs-Lab/lmms-eval?
- Unified evaluation toolkit for benchmarking multimodal large language models and vision-language models across diverse task types.
- Is lmms-eval open source?
- Yes — EvolvingLMMs-Lab/lmms-eval is an open-source project tracked on heatdrop.
- What language is lmms-eval written in?
- EvolvingLMMs-Lab/lmms-eval is primarily written in Python.
- How popular is lmms-eval?
- EvolvingLMMs-Lab/lmms-eval has 4.3k stars on GitHub.
- Where can I find lmms-eval?
- EvolvingLMMs-Lab/lmms-eval is on GitHub at https://github.com/EvolvingLMMs-Lab/lmms-eval.