← all repositories

datawhalechina/diy-llm

A systematic Jupyter Notebook course teaching how to build large language models from scratch, covering pretraining, alignment, and distributed training.

876 stars Jupyter Notebook LearningLanguage ModelsLLMOps · Eval
diy-llm
Velocity · 7d
+4.5
★ / day
Trend
steady
star history

The course provides a structured curriculum for building LLMs from the ground up. It covers core components including tokenizers, transformer architectures, and mixture-of-experts. Students progress through six hands-on assignments covering distributed training, RLHF/SFT alignment techniques, and GPU programming with CUDA/Triton. The material includes inference optimization and scaling laws, offering a complete full-stack understanding of LLM development.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.