Systematically rude to language models, for science

What it does

garak is a command-line scanner that bombards a target language model with probes—static, dynamic, and adaptive—to trigger failures like hallucinations, toxic outputs, data leakage, and jailbreaks. It reports results in a simple pass/fail format with failure rates, and supports a wide range of targets from local llama.cpp weights to commercial APIs such as OpenAI, AWS Bedrock, and NVIDIA NIM. The README explicitly likens it to nmap or Metasploit, only the attack surface is prompt engineering rather than open ports.

The interesting bit

The README highlights a counter-intuitive result: a newer GPT variant proved more susceptible to encoding-based prompt injection than an older one, illustrating that model “improvement” does not equal hardened security. garak’s value is in surfacing these regressions automatically rather than relying on vibe checks.

Key highlights

• Broad backend support: Hugging Face, OpenAI, Bedrock, Replicate, Cohere, Groq, LiteLLM, REST endpoints, and local GGUF models via llama.cpp.
• Combines static, dynamic, and adaptive probes rather than a fixed test suite.
• Clear CLI output: each probe gets a progress bar, then a row with failure rates; hits are marked FAIL.
• Includes a basic log analyzer script to surface the most successful prompts.
• Published by NVIDIA, with an arXiv paper and DEF CON slides backing the methodology.

Caveats

• Developed primarily on Linux and macOS; Windows is tested via CI but not the native dev environment.
• The probe table in the README is truncated, so the full inventory of available tests isn’t visible without installing or browsing the docs.
• The bundled analysis script is described as “basic,” suggesting you’ll likely need your own parsing for deep incident investigation.

Verdict

Security engineers and LLM operators who need empirical evidence of model weaknesses should grab this; if you are looking for a polished GUI or a compliance checkbox tool, this is not it.