OpenAI's open-source tool for stress-testing your LLM before users do

What it does

promptfoo evaluates prompts, agents, and RAG pipelines against automated test suites, then surfaces the results in a local web viewer or the terminal. It also runs red-team scans and vulnerability checks against LLM apps, generating security reports. Everything is driven by declarative config files meant to live in version control and CI/CD.

The interesting bit

The project treats prompt engineering as infrastructure rather than art: you define test cases, compare model outputs side-by-side, and gate deploys on measurable scores. The red-teaming angle is genuinely unusual for an evaluation framework—most tools optimize for accuracy; this one also probes for jailbreaks, data leakage, and policy violations.

Key highlights

• Runs fully offline; prompts and evals never leave your machine
• Supports OpenAI, Anthropic, Azure, Bedrock, Ollama, and other providers
• Ships as npm package, brew formula, and pip install
• Includes live reload, caching, and a local web UI for result comparison
• Now part of OpenAI, but remains MIT-licensed and open source
• Claims to power production LLM apps serving 10M+ users

Caveats

• The “10M+ users” production claim is stated without supporting detail
• Most LLM providers still require API keys, so “fully offline” applies to the eval tooling, not necessarily the model inference

Verdict

Worth a look if you’re shipping LLM features to production and tired of manual prompt tweaking. Skip it if your AI use is limited to occasional ChatGPT queries with no regression-testing needs.