AgentSafeLabs/safelabs-eval
An open-source red-teaming and evaluation framework for AI agents built around an OWASP-inspired ASI01–ASI10 security taxonomy.
Collecting fresh signals — velocity needs a few days of history.
collecting data…
star history
safelabs-eval tests AI agents for security vulnerabilities by firing 300 curated adversarial prompts across 10 agent-security categories at HTTP agent endpoints or wrapped Python callables. It scores responses with pattern-based detectors, requiring no LLM calls or agent code modifications, and produces structured security reports. It supports agents built with frameworks such as LangChain, CrewAI, AutoGen, LlamaIndex, and the OpenAI Agents SDK.
Frequently asked
- What is AgentSafeLabs/safelabs-eval?
- An open-source red-teaming and evaluation framework for AI agents built around an OWASP-inspired ASI01–ASI10 security taxonomy.
- Is safelabs-eval open source?
- Yes — AgentSafeLabs/safelabs-eval is open source, released under the Apache-2.0 license.
- What language is safelabs-eval written in?
- AgentSafeLabs/safelabs-eval is primarily written in Python.
- How popular is safelabs-eval?
- AgentSafeLabs/safelabs-eval has 570 stars on GitHub.
- Where can I find safelabs-eval?
- AgentSafeLabs/safelabs-eval is on GitHub at https://github.com/AgentSafeLabs/safelabs-eval.