Meta's purple-team play: attack benchmarks and defenses, one repo

What it does Purple Llama is Meta’s umbrella project for generative-AI trust and safety, and it has two halves. The defensive half ships safeguards that wrap a model: Llama Guard 3 for input/output content moderation, Prompt Guard for injections and jailbreaks, and Code Shield for filtering insecure LLM-generated code at inference time. The offensive half is CyberSec Eval, a benchmark suite (now on v3) that measures how readily a model suggests insecure code, complies with malicious requests, or falls for prompt injection.

The interesting bit The name is the thesis: purple teaming means running red (attack) and blue (defense) postures together, and this repo does both — the same project ships the evals that expose a model’s weaknesses and the filters meant to catch what slips through. CyberSec Eval 3 also goes further than most safety benchmarks, testing spear phishing capability and autonomous offensive cyber operations rather than just tone.

Key highlights

• Llama Guard 3 models are fine-tuned from Llama 3.1/3.2, target the MLCommons hazards taxonomy, and add 7 languages, a 128k context window, and image reasoning.
• Prompt Guard distinguishes prompt injection (untrusted third-party data hijacking the context window) from jailbreaks (instructions overriding built-in safety).
• Code Shield filters insecure code at inference time and is MIT licensed, unlike the guard models.
• CyberSec Eval is grounded in CWE and MITRE ATT&CK, with a public leaderboard on Hugging Face; Meta claims v1 was the first industry-wide cybersecurity eval suite for LLMs.
• Meta explicitly ties the work to the White House voluntary commitments on responsible AI — metrics for quantifying LLM cyber risk.

Caveats

• It’s an umbrella, not a monolith: model weights live on Hugging Face and onboarding material is deferred to the separate llama-recipes repo, so expect some repo-hopping.
• Licensing is split — benchmarks and Code Shield are MIT, but the Guard models carry Llama Community licenses. “Permissive” is doing some work in that sentence.
• The README promises more components “over time”; parts of the roadmap are still aspiration.

Verdict Worth a look if you’re shipping an LLM application and need moderation, injection defenses, or a defensible number for your model’s cyber risk. Less useful if you want one uniform license or a self-contained toolkit — this is a hub, with the heavy lifting distributed across other repos.