When GPT gets a black hoodie and a Kali Linux ISO

What it does

PentestGPT is an autonomous penetration-testing agent that runs inside Docker and uses LLMs to solve security challenges end-to-end. You point it at a target IP, optionally add context like “WordPress site, focus on plugin vulnerabilities,” and it plans, executes tools, and chases flags while you watch in a TUI. It handles web, crypto, reversing, forensics, PWN, and privilege escalation.

The interesting bit

The v1.0 rewrite is “agentic” — session persistence lets you pause and resume long engagements, and the tool routes different cognitive loads to different models: a lightweight model for background tasks, a heavier coder model for reasoning, another for long context. The authors benchmarked it on 104 XBOW challenges and published the results at USENIX Security 2024.

Key highlights

• 86.5% success rate (90/104) on the XBOW benchmark suite, averaging $1.11 and 6.1 minutes per solved challenge
• Docker-first deployment with security tools pre-installed; local LLM support via LM Studio or Ollama
• Multi-category coverage: web exploitation, cryptography, reverse engineering, forensics, binary exploitation, privilege escalation
• Anonymous telemetry via Langfuse, with explicit opt-out flags and environment variables
• Legacy v0.15 branch retains broader multi-provider support (OpenAI, Gemini, DeepSeek, Ollama)

Caveats

• Multi-model support for OpenAI and Gemini is listed as “in progress” for v1.0; current release is heavily Anthropic/Claude-oriented
• The benchmark numbers are self-reported on a specific CTF suite; real-world target complexity will vary

Verdict

Red teamers and CTF addicts who want an AI teammate to grind boxes overnight should look here. If you need a polished enterprise pentest reporting suite or are allergic to Docker, this is not your tool.