A debugger that lets you intervene in GPT-2's forward pass

What it does

Transformer Debugger (TDB) is a React-based tool for dissecting the internals of small language models—specifically GPT-2 and its autoencoders. It lets you ask why a model emitted one token over another or why an attention head fixated on a specific token, then surfaces the responsible neurons, attention heads, and sparse autoencoder latents. The system auto-generates explanations of what causes those components to activate most strongly and traces connections between them to help discover circuits.

The interesting bit

You can intervene in the forward pass and immediately see how the model’s behavior shifts, all before writing any code. By combining automated interpretability techniques with sparse autoencoders inside a React frontend, TDB turns circuit investigation from a scripting exercise into a point-and-click exploration.

Key highlights

• React frontend (Neuron Viewer) that hosts the debugger and dedicated pages for individual MLP neurons, attention heads, and autoencoder latents.
• Python backend (Activation Server) that runs inference on a subject model and serves activation datasets from public Azure buckets.
• Includes a simple inference library for GPT-2 models and their autoencoders, with hooks to grab activations.
• Ships with collated activation datasets showing top-activating examples for neurons, heads, and autoencoder latents.
• Developed by OpenAI’s Superalignment team for rapid exploration before needing to write code.

Caveats

• Explicitly scoped to small language models; the README centers on GPT-2 and does not mention support for larger or different architectures.

Verdict

Worth a look if you are investigating specific behaviors in GPT-2 and want to trace circuits without writing code first. Skip it if you need to audit large-scale models or prefer a purely programmatic workflow.