Why Cloudflare Open-Sourced the Seed of Its AI Vulnerability Harness

The release arrives as enterprise security faces a numbers problem. One industry survey noted that 94 percent of cloud customers faced cyber threats in 2023, with over 60 percent compromised, while the cloud computing market is projected to expand from roughly $752 billion in 2024 to approximately $2.3 trillion by 2030. Data security and shared responsibility remain the biggest challenges for organizations, yet security programs are struggling to keep pace with adoption speed and AI-driven workload complexity. Against that backdrop, Cloudflare’s publication of the security-audit-skill repository reads less like a tool drop and more like an architectural thesis: what if the auditor itself were automated, parallelized, and adversarial?

This is the single-repo seed that grew into Cloudflare’s internal multi-stage, fleet-wide vulnerability discovery harness, described in the company’s blog post Build your own vulnerability harness. Open-sourcing it offers a rare look at how a major infrastructure provider is restructuring security auditing around agentic AI rather than traditional scanners. The repository does not contain a productized platform; it contains a workflow grammar for turning a coding agent into a red team that can argue with itself.

Most AI security assistants today function like accelerated static analyzers. A skill listed on the Claude Directory, for example, scans for OWASP Top 10 patterns, known CVEs, hardcoded secrets, and misconfigurations—valuable, but fundamentally pattern-matching. Cloudflare’s skill is structurally different. It is a six-phase pipeline that treats the coding agent not as a scanner, but as a distributed audit team with assigned roles.

The workflow begins with reconnaissance: parallel research agents map the application’s architecture, trust boundaries, and input surfaces, producing a living document of the target. This mimics the planning stage of traditional security audit services, where scope, critical assets, and compliance requirements are defined before execution. By forcing a mapping phase first, the skill reduces the tendency of large language models to fixate on superficial patterns without context.

Then hunting agents attack from seven angles—injection, access control, business logic, cryptography, feature abuse, chained attacks, and a wildcard—each able to spawn sub-agents to dig deeper. This is not a monolithic scan. The methodology explicitly covers “obvious things” via dedicated prompts, acknowledging that LLMs, like junior auditors, sometimes miss what is staring at them. The skill blurs the line that conventional definitions draw between audits and penetration tests, taking the systematic review structure of the former and the active exploitation mindset of the latter.

The most distinctive phases are validation and independent verification. After hunting, separate agents attempt to disprove each finding. The agent that validates a bug is never the agent that found it. This is a direct architectural response to LLM hallucination. Rather than trusting a single model pass, the skill institutionalizes distrust. Findings that survive advance to reporting, which produces both human-readable narratives and detailed traces for medium-and-above issues. The structured output is then schema-validated and subjected to independent verification, where fresh agents check every factual claim against the actual source code. This triple-check—hunter, skeptic, verifier—mirrors the separation of duties that human audit teams strive for but rarely achieve at this granularity.

The design principles underlying these phases are deliberately conservative. The skill demands that every finding include a concrete attack scenario; theoretical flaws are discarded. Severity is judged by likelihood times impact, not deviation from a checklist. Defense-in-depth gaps are noted as hardening opportunities, not vulnerabilities, provided an upstream layer prevents exploitation. And the skill admits its own incompleteness: testing shows a single run finds roughly half the total vulnerabilities across multiple runs, so each run reads prior findings.json files to skip known issues and target gaps. Coverage is built additively, not exhaustively.

Inside Cloudflare, this seed evolved into a multi-stage harness orchestrated across repositories at scale. The open-sourced version strips away the fleet orchestration, leaving the core workflow exposed. That reduction is intellectually honest—it prevents the community from mistaking a corporate platform for a community tool—but it also means adopters must supply their own scheduling, result aggregation, and fleet management.

Traditional security audit software has moved aggressively toward continuous monitoring, API-driven evidence collection, and compliance automation. These platforms integrate with cloud infrastructure, identity providers, and DevOps pipelines, centralizing evidence and reducing audit preparation time. Modern compliance automation tools similarly aim to cut repetitive work and maintain audit-ready status. These systems excel at configuration drift, permission sprawl, and policy enforcement.

Cloudflare’s skill ignores that stack entirely. It has no dashboard, no CI/CD hook, and no SOC 2 mapping. Instead, it occupies a different niche: reasoning about code that pattern-based tools cannot see. Business logic flaws, chained exploit paths, and feature abuse require contextual judgment, not signature matching. Where a conventional tool might flag a SQL concatenation as a potential injection, this skill asks whether that concatenation is reachable, whether input validation elsewhere neutralizes it, and whether the resulting exploit has real impact. It is closer to a penetration tester’s mindset than a compliance auditor’s checklist.

This distinction matters because the industry is bifurcating. One path automates compliance and posture management; the other must automate expertise and reasoning. The Cloudflare skill is an early, open-source stake in the second path. It does not replace CNAPPs or vulnerability management platforms; it addresses the expertise gap that those platforms cannot fill—the human judgment required to distinguish a theoretical flaw from an exploitable vulnerability.

The repository is candid about its limitations. It is explicitly the “single-repo starting point” for a system that eventually became fleet-wide inside Cloudflare. Running it requires a coding agent capable of tool use and parallel sub-agents, which immediately limits deployment to models with advanced agentic features. A Node.js validator handles schema compliance for structured output, but the heavy lifting remains LLM-dependent. And while the adversarial validation layer reduces false positives, it does not eliminate them; the entire system is ultimately a reasoning engine operating on probabilistic outputs.

There is also an epistemological tension. The skill demands concrete attack scenarios, yet it operates on static source code without runtime instrumentation. A human red team might pivot from a subtle logic bug to a remote shell by observing application behavior; an LLM agent must infer that path. The skill compensates by requiring detailed traces and independent verification, but the gap between static reasoning and dynamic exploitation is a hard boundary. It finds exploitable vulnerabilities in theory; proving exploitability in practice still requires human confirmation or dynamic testing.

Moreover, the additive coverage model implies that the skill is non-deterministic and coverage is probabilistic. This is both a feature—it explores diverse code paths across runs—and a limitation—it offers no guarantee of completeness. For teams accustomed to compliance audits with binary pass/fail outcomes, this probabilistic approach may require a shift in expectations.

What Cloudflare has released is less a product than a procedural framework for distrusting AI. By open-sourcing the seed of its internal harness, it invites other organizations to experiment with agentic audit architectures. The broader context—cloud markets expanding toward $2.3 trillion, security expertise failing to scale, and manual audits collapsing under retroactive checks and static spreadsheets—suggests that such experiments are necessary.

The skill points toward a future where security auditing is not performed by a tool or a human, but by a structured argument between machines. Whether that future produces fewer false positives or merely automates confusion at scale remains an open question. For now, the repository stands as one of the most concrete attempts to build skepticism into the prompt layer rather than hoping it emerges from the model.