A test bench for the oldest trick in the LLM book

What it does

This is a benchmark toolkit from Duke researchers for prompt injection: it lets you compose a target task (say, sentiment analysis on sst2), an injected task (say, spam detection), an attack strategy, an LLM, and a defense, then measure the attack success via a computed ASV. Everything is wired through config files and factory methods, so swapping one variable at a time is the whole point. It backs three papers, including a USENIX Security 2024 paper formalizing the attack/defense space.

The interesting bit

Beyond benchmarking, it ships two fine-tuned defense components: DataSentinel, a detector trained game-theoretically against adaptive attackers, and PromptLocate, which doesn’t just flag a contaminated prompt but localizes the injected text and recovers the clean data. Detection-then-localization as a pipeline is a more surgical answer than “refuse everything.”

Key highlights

• Composable experiment matrix: attacks, defenses, target/injected tasks, and models are all pluggable via configs
• run.py supports a --dry-run mode and config fingerprints on response archives, so cached results are only reused when the experiment actually matches
• ASV (attack success value) computed by a built-in evaluator for apples-to-apples comparisons
• DataSentinel and PromptLocate checkpoints available for download; more detectors and fine-tuning code promised
• Companion slides from a Safer with Google Summit 2025 talk for a deeper conceptual tour

Caveats

• The demo configs point at PaLM2, which is showing its age; Llama and GPT configs exist but you bring your own API keys
• Fine-tuning code for the detectors is “coming soon” — you can use the checkpoints but not yet reproduce them
• Requires a conda environment per the README; no lighter install path is offered

Verdict

If you research LLM security or need a defensible evaluation harness for injection defenses, this is one of the few properly benchmarked options. If you just want a quick guardrail for a production app, the checkpoint downloads and API-key plumbing make it more lab than library.