braintrustdata/autoevals
AutoEvals is a library for evaluating AI model outputs using LLM-as-a-judge, heuristic, and statistical methods.
Collecting fresh signals — velocity needs a few days of history.
collecting data…
star history
AutoEvals provides automatic evaluation methods for AI model outputs, including LLM-as-a-judge prompts adapted from OpenAI’s evals, heuristic metrics like Levenshtein distance, and statistical metrics like BLEU. It supports both Python and TypeScript, allows custom model-graded evaluations with configurable prompts, and integrates with OpenAI’s API for grading completions. The library is developed by Braintrust and targets developers building LLM applications who need to assess factuality, safety, and other subjective qualities.
Frequently asked
- What is braintrustdata/autoevals?
- AutoEvals is a library for evaluating AI model outputs using LLM-as-a-judge, heuristic, and statistical methods.
- Is autoevals open source?
- Yes — braintrustdata/autoevals is open source, released under the MIT license.
- What language is autoevals written in?
- braintrustdata/autoevals is primarily written in Python.
- How popular is autoevals?
- braintrustdata/autoevals has 1k stars on GitHub.
- Where can I find autoevals?
- braintrustdata/autoevals is on GitHub at https://github.com/braintrustdata/autoevals.