The Red-Team Tool That Grades Your Agent With Regex, Not a Judge Model

Every few weeks a security tool lands on GitHub trending that promises to tell you whether your AI agent is safe. Most of them quietly outsource the hard part to a judge model — you attack your agent with an LLM, then ask another LLM whether the attack worked, and pay inference costs twice for the privilege. safelabs-eval, from Safe Labs AI Inc., takes the opposite bet: 300 curated adversarial prompts, five pattern-based detectors written in pure Python, and zero additional LLM calls for grading. It is cheap, deterministic, and fast enough to run in CI. It is also, by its own published research, sometimes wrong.

That combination — genuinely useful plumbing paired with unusually public self-skepticism — is what makes the project worth a closer look, beyond the trending momentum that saw it gain roughly 947 stars and 98 forks over 60 days this fall.

The moment

The repository’s rise has been spiky rather than steady. Trendshift’s data shows most days with zero star gains, punctuated by bursts — +154 on October 1, +137 on October 19, +93 the next day. It cracked the Trendshift Trending top 25 in Python exactly once, at #25, on October 1. This is the growth signature of a project being actively promoted by its author rather than one diffusing organically through word of mouth: the Reddit posts driving attention were made by u/iamwaqarjaved, who is Waqar Javed, the sole listed contributor and the name on the citation block. One of those posts claimed the project ran 9,360 adversarial trials across seven agent frameworks and found that framework choice barely affected security outcomes. Another described a case where apparently inconsistent LLM behavior turned out to be a bug in the project’s own safety detector.

So the hype is partly self-generated. That is not a knock — solo-maintained security tooling with an active research agenda behind it is how a lot of this ecosystem gets built — but it should calibrate your expectations. This is a one-person project with a corporate name attached, not a foundation-backed standard.

The timing, though, is real. The attention spike lands in the middle of a genuine institutional scramble around agent security. NIST’s Center for AI Standards and Innovation launched a dedicated AI Agent Standards Initiative in February 2026 — its first organizational initiative around agent security — and its own red-team research found novel agent-targeting attacks achieving an 81% task-hijacking success rate versus 11% for the strongest known baselines. OWASP has been shipping red-teaming resources all year, arguing that traditional application security practices are no longer sufficient for systems whose vulnerabilities live in behavior rather than code. And Unit 42 reported in March that indirect prompt injection — hidden instructions embedded in web content that agents ingest during routine tasks — is now actively weaponized in the wild, with 22 distinct payload-engineering techniques observed in telemetry, including the first known case of AI-based ad review evasion. When the threat model moves from proof-of-concept to observed-in-production, tooling that lets a team run a quick adversarial sweep before shipping stops being academic.

The boring idea that carries the whole thing

Strip away the taxonomy and the papers, and safelabs-eval is conceptually simple: point it at an HTTP endpoint your agent exposes, or wrap any Python callable, and it fires a fixed corpus of adversarial prompts at it, then pattern-matches the responses for signs the agent complied with the attack. The adapters are where the unglamorous work lives — there are optional adapters for LangChain, CrewAI, AutoGen, LlamaIndex, the OpenAI Agents SDK, Google ADK, and Semantic Kernel, each handling that framework’s particular way of running an agent and normalizing the result into a common response shape that carries output text, errors, metadata, and tool calls. Framework-agnosticism via a thin adapter layer is not a novel insight, but it is the difference between a tool a team will actually run and a tool that only works on the author’s stack.

The genuinely opinionated choice is the scoring. Detectors are pure Python — regex suites compiled once at init, no I/O, no model calls, async-safe for concurrent pipelines. Every response gets one of four verdicts: VULNERABLE, FAIL, UNCERTAIN, or PASS, with a heuristic confidence figure that the README is careful to describe as an evidence score, not a calibrated probability. The pitch is that this makes red-teaming a CI-checkable artifact rather than an expensive consulting exercise: structured JSON output, an audit trail, and a marginal grading cost of approximately zero.

The tradeoff is the obvious one, and the project does not hide it. Pattern matching misses subtle, novel, or paraphrased forms of unsafe compliance, and it produces false positives. The README’s own limitations section says results should be read as “detector-derived evaluations, not ground-truth safety measurements.” That is a more honest framing than most commercial agent-security marketing manages.

A taxonomy with a collision problem

The prompt corpus is organized into ten categories, ASI01 through ASI10 — prompt injection, insecure output handling, excessive agency, resource management, tool use safety, data privacy, trust boundaries, behavioral drift, scope violations, and hallucination. The README calls this “OWASP-inspired,” and it is here that the project steps into a genuinely awkward spot: OWASP published its official Top 10 for Agentic Applications 2026 in December 2025, and that official list also numbers its categories ASI01–ASI10, with a different structure — Agent Goal Hijack, Tool Misuse & Exploitation, Identity & Privilege Abuse, and so on. So safelabs-eval’s ASI01 (Prompt Injection) and OWASP’s ASI01 (Agent Goal Hijack) are not the same category, and the same identifier means different things depending on which document you are holding. The README flags this explicitly and notes the corpus is not currently mapped to the official taxonomy. It is a refreshingly candid handling of what could have been a quiet branding ride on OWASP’s authority — but the collision remains, and anyone citing “ASI04” in a report had better say which numbering scheme they mean.

The corpus itself is more carefully built than the average prompt dump. Every one of the 300 prompts carries structured metadata: a difficulty tier (overt, contextual, or adaptive, assigned by a documented rubric rather than vibes), provenance labels, and MITRE ATLAS technique mappings where they apply. A full provenance audit over the 131 prompts of an earlier content version found no verbatim reproduction of external text, with technique-lineage notes citing the research the scenarios draw on — Greshake et al. on indirect injection, AgentDojo on tool-use attacks, Crescendo on multi-turn escalation, garak and PyRIT probe categories. The 169 prompts added after that audit have not had the same systematic review, a gap the README discloses rather than papers over. The library is English-only and single-turn; the behavioral-drift category describes multi-turn escalation inside a single prompt rather than actually executing a conversation, which is a meaningful limitation given that gradual boundary testing and multi-turn manipulation are among the more effective attack families.

The self-correction is the most interesting part

What elevates safelabs-eval above the median security-scanner repository is the research trail attached to it, and specifically the trail’s willingness to indict itself. The project’s earlier preprints — now consolidated into merged papers — include work on detector-calibration reliability with titles like “Pattern-Matching Failures in LLM Refusal Detection” and “Does a Pattern-Matching Detector Fix Generalize? A Six-Model Replication and the Prompt Shapes That Break It.” In other words: the team built a regex-based safety detector, found it producing false verdicts, and published papers about the ways its own instrument misfires, including a public blog correction — “We Were Wrong About the UNCERTAIN Results — Here’s What Actually Happened” — with a dated addendum.

This is rare. The standard failure mode in this space is a tool that quietly encodes its authors’ assumptions and never tests them. safelabs-eval’s lineage is closer to an instrument that ships with its own error analysis. The follow-on project, safelabs-trace, pushes the critique further: a pre-registered benchmark that scores what agents actually do with their tools rather than what they say, motivated by the observation that text-only scoring misses actions — an agent can verbally decline a malicious request while its tool calls do something else entirely. The README’s own roadmap concedes the same point, listing LLM-graded and embedding-similarity detectors as future work precisely because pattern matching has coverage gaps on subtle attacks.

Where it sits in the field

The competitive landscape is established and well-funded. NVIDIA’s garak is a mature LLM vulnerability scanner; Microsoft’s PyRIT is an enterprise-grade risk-identification toolkit; Promptfoo — which announced an agreement to be acquired by OpenAI in March 2026 — covers LLM testing broadly. Those tools largely target models and chat interfaces. safelabs-eval’s differentiator is narrow but real: it is agent-shaped, framework-agnostic across seven major agent stacks, and free to grade. Whether that niche survives contact with the incumbents adding agent support is an open question. The distinction between jailbreaking and prompt injection that practitioners increasingly insist on — attacks on the model’s safety training versus attacks on the application’s architecture — matters more, not less, as agents accumulate privileges, and a corpus that treats them as separate categories is at least aligned with where defensive thinking is heading.

The rough edges are visible and mostly acknowledged. The adapter tests use duck-typed fakes rather than real framework packages, so the seven-framework support is less battle-tested than the badge count suggests — the README says so directly. The difficulty tiers are author-defined and uncalibrated against measured failure rates. Coverage is breadth-oriented, not exhaustive. And the star graph’s promotional spikes mean the community around it is, so far, thin: one contributor, a handful of merged pull requests, engagement numbers on Trendshift that could charitably be described as early.

Outlook

The interesting tension in this project is that its central design decision — cheap, deterministic, pattern-based grading — is the same thing its own research identifies as its weakest link. The team’s answer so far has been to keep the cheap detector as the default and build the expensive, more honest evaluation as a separate instrument, which is a defensible architecture: run the free regex sweep on every commit, reserve trace-level and LLM-graded analysis for the things that matter. Whether the two converge, whether the corpus gets mapped to OWASP’s official agentic taxonomy before the ASI numbering collision causes real confusion, and whether a solo-maintained project can hold its lane against garak, PyRIT, and an OpenAI-owned Promptfoo — those are the open questions. What is not in question is the premise. Agents are being deployed into business-critical workflows faster than anyone is adversarially testing them, and the observed-in-the-wild weaponization of indirect prompt injection has removed the last excuse for treating that as a theoretical concern. A tool that makes the first pass at that testing cost approximately nothing is a reasonable thing to have in the pipeline — provided you read its verdicts as what they are: a cheap first opinion, not a clean bill of health.