Peel back the layers of a live GPT in 3D

What it does

llm-viz renders a working GPT-style transformer in 3D as it performs inference. The default demo runs a tiny minGPT model—lifted from Andrej Karpathy’s implementation—that sorts the letters A, B, and C, letting you observe the matrix operations in real time. The renderer can handle arbitrary network sizes, including GPT-2 topologies, though the weights are not bundled.

The interesting bit

Most LLM explainers are static diagrams; this is a live, spatial model of the machinery where you can watch the computation unfold in three dimensions. It also shares a repo with an unreleased 2D CPU schematic editor, suggesting the author has a habit of making invisible hardware and software visible.

Key highlights

• Live 3D visualization of a real minGPT model sorting letters in the browser.
• Renderer scales to arbitrary network sizes, including GPT-2, but weights are omitted to avoid multi-hundred-megabyte downloads.
• Bundled in a monorepo with the author’s homepage and shared TypeScript utilities.
• A work-in-progress 2D digital circuit editor for RISC-V CPU simulation lives in the same repo, though it is not yet exposed.

Caveats

• The repo is a deployment monorepo mixing the author’s personal homepage, the LLM visualizer, and an unreleased CPU simulator, so project boundaries are fuzzy.
• Only the tiny sorting model includes working weights; larger configurations require you to source weights separately.
• The CPU simulation is marked WIP and inaccessible in the current build.

Verdict

A solid stop for visual learners who want to see transformer inference rather than read about it. Skip if you need training code or a production inference server.