Jupyter notebooks that reverse-engineer inference engines

What it does

InfraTech is a curated collection of Jupyter notebooks that teach AI infrastructure through code. It covers LLM inference mechanics, training frameworks, distributed systems basics, and hardware-aware optimizations. Each notebook pairs with a detailed Zhihu article, walking through implementations of schedulers, attention variants, and memory management strategies used in production engines like vLLM and SGLang.

The interesting bit

Rather than wrapping frameworks in abstraction, the repo has you rebuild their guts: hand-crafting a vLLM-style scheduler, dissecting SGLang’s RadixAttention, and benchmarking KV cache trade-offs. It treats inference engines as mechanical puzzles to be taken apart, not black boxes to be installed.

Key highlights

• 20+ notebooks spanning inference frameworks (vLLM, SGLang, Nano-vLLM), distributed training, and quantization
• Difficulty-rated exercises from basic sampling to multi-dimensional parallel strategies (DP/TP/PP/SP/EP)
• Hands-on implementations of production mechanisms: chunked prefill, speculative decoding, CUDA graphs, and LoRA serving
• Cross-referenced with detailed Chinese-language articles for theory behind the code
• Covers RL training-inference colocation and weight hot-swapping between Megatron and SGLang

Caveats

• All explanatory text and linked articles are in Chinese
• Not a unified software package; it is educational material and reference notebooks
• Some notebooks depend on external repositories (e.g., BasicCUDA) for specific exercises

Verdict

Worth bookmarking if you are an engineer breaking into LLM inference or AI infrastructure and you read Chinese. Skip it if you need a drop-in library or expect polished, English-language documentation.