Hand-write Flash Attention in your browser, graded offline

What it does

Pyre Code is a self-hosted, browser-based practice platform for implementing modern ML internals. It serves 76 problems—ranging from basic activations like ReLU to frontier architectures like Mamba, MLA, and flow matching—through a Monaco editor. A local FastAPI grading service runs tests against your Python submission and returns pass/fail per case in seconds, all without a GPU.

The interesting bit

The curriculum reads like a disassembly manual for today’s AI stack: you write your own LayerNorm, KV cache, speculative decoding, DPO loss, and Paged Attention, then compare against reference solutions. It is essentially a curated LeetCode for the guts of Transformers, vLLM, TRL, and diffusion models, with learning paths that thread fundamentals into full training pipelines.

Key highlights

• 76 problems across attention variants, inference kernels, training tricks, alignment algorithms, GNNs, and diffusion
• Runs fully offline: browser editor, local FastAPI grader, SQLite progress tracking
• Optional AI Help via any OpenAI-compatible API, configured server-side or per-user
• Reference solutions and submission history for reviewing failed attempts
• Built on the torch_judge engine from the TorchCode project

Verdict

Worth your time if you’re prepping for ML systems interviews or you learn best by writing code rather than watching lectures. Skip it if you want high-level API tutorials; this is for people who want to know why nn.MultiheadAttention works.