Stanford’s LLM course, rebuilt as a Chinese coding workshop

What it does

Diy-LLM is a Chinese-language course that reconstructs Stanford’s CS336 into a 15-chapter, six-assignment workshop. It walks learners through building a language model from scratch—tokenizer, Transformer, MoE, CUDA/Triton kernels, distributed training, scaling laws, data pipelines, and alignment via SFT and GRPO—using notebooks and prose aimed at Chinese hardware constraints and open-source ecosystems like Qwen and DeepSeek.

The interesting bit

Instead of a straight translation, the authors treat the original as raw ore and smelt it into a local “alchemy workshop.” They explicitly replace Western-centric examples with domestic models and cloud realities, and they force you to write FlashAttention-2 in Triton and fit scaling laws before you can call yourself done.

Key highlights

• Six progressive assignments: hand-roll a minimal LLM, optimize kernels, process Common Crawl data, run distributed training, apply RLVR/GRPO for math reasoning, and benchmark with lm-evaluation-harness.
• Heavy systems focus: dedicated chapters on GPU programming, CUDA, and distributed training, not just model architecture.
• Localized context: explicitly references Qwen and DeepSeek, and acknowledges that Chinese learners face different network and compute constraints.
• Most chapters and assignments are marked complete, though Chapter 1 and 15 remain works in progress.

Caveats

• The material is entirely in Chinese; English-only readers need not apply.
• A few chapters (e.g., Chapter 1 on tooling, Chapter 15 on extensions) are still pending or marked updating.
• Full training runs require GPU access; the authors admit CPU-only learners can only debug, not finish the coursework.

Verdict

Grab this if you are a Mandarin-speaking developer with PyTorch experience who wants to stop reading LLM papers and start forging weights. Skip it if you are looking for a drop-in library or a quick API tutorial.