Build LLaMA2 from scratch: a Chinese zero-to-LLM course

What it does Happy-LLM is an open-source Chinese textbook-style courseware repo that walks you from NLP basics through Transformer math to training a real 215-million-parameter LLaMA2 model. It splits the journey into theory (chapters 1–4) and practice (chapters 5–7), covering pre-training, supervised fine-tuning, LoRA/QLoRA, and downstream applications like RAG and Agents. The project also ships actual checkpoint files and a companion PDF, so you can read on a tablet and train on a GPU.

The interesting bit Most tutorials stop at “call AutoModelForCausalLM”; this one makes you write the PyTorch layers yourself in Chapter 5 before letting you use the Hugging Face transformers framework in Chapter 6. That deliberate regression—from hand-rolled tensors to industrial tooling—mirrors how you actually learn the stack.

Key highlights

• Hand-builds a LLaMA2 architecture from raw PyTorch, including tokenizer training and a 215M-parameter pre-trained checkpoint.
• Covers the full modern training stack: pre-training, SFT, and parameter-efficient fine-tuning with LoRA/QLoRA.
• Includes applied chapters on model evaluation, RAG retrieval augmentation, and simple Agent implementations.
• Publishes ready-to-download PDF slides and ModelScope checkpoints for both base and SFT versions.
• Accepts community blog posts and notes via an open “Extra Chapter” PR process.

Caveats

• The content is overwhelmingly Chinese; the English README exists but the curriculum and primary materials are not translated.
• One training sub-topic (preference alignment) and the Extra Chapter blog collection are explicitly marked as work-in-progress.
• Chapter dependencies are isolated, so expect to juggle multiple Python environments if you run the code sequentially.

Verdict Ideal for Mandarin-speaking students, researchers, or hobbyists who want to close the gap between “I read the Attention Is All You Need paper” and “I have a fine-tuned model on my GPU.” If you are looking for a drop-in training framework or English-language prose, this is a tutorial, not a tool.