The anatomy textbook for LLMs and reinforcement learning

What it does This repository is a curated visual reference library rather than a framework. It hosts more than a hundred original architecture diagrams—released as scalable SVGs and PNGs—that walk through the internals of large language models, vision-language models, and reinforcement learning algorithms. The author, who wrote a Chinese textbook on the subject, uses the repo to map out everything from Transformer input layers to the full RLHF training loop, covering SFT, DPO, PPO, GRPO, and CoT distillation.

The interesting bit Instead of recycling standard paper figures, the author redraws concepts into unified “maps,” including what they describe as the largest publicly available diagram of RL algorithms and a complete structural overview of LLMs. The SVGs contain live text, so you can zoom infinitely and copy labels, which turns a static reference into something closer to a searchable textbook.

Key highlights

• Covers the full modern LLM training stack: pre-training, SFT (LoRA, Prefix-Tuning), DPO, RLHF/RLAIF, and inference tricks such as RAG, CoT, and beam search.
• Includes foundational RL theory—Policy Gradient, Actor-Critic, DQN, TRPO, PPO—alongside its LLM-specific applications like GRPO and rejection sampling.
• Diagrams are published as vector graphics with selectable text rather than flattened screenshots.
• Tied to a published Chinese book; the repo serves as a living appendix that the author updates with corrections and new figures.

Caveats

• The project is almost entirely static images and explanatory markdown; do not expect runnable training code or importable libraries.
• The primary labels and detailed commentary are in Chinese, though an English README is linked.

Verdict Grab this if you are a visual learner trying to make sense of modern LLM training pipelines or RL fundamentals. Skip it if you are hunting for a training framework or copy-pasteable implementations.