A brutalist reading list for AI inference performance

What it does

This repository is a tightly structured syllabus for learning GPU performance engineering and production inference. It arranges papers, official documentation, and reference implementations along the path of a real inference request—from a single CUDA warp through kernel optimization, inference engines, and distributed systems.

The interesting bit

The curation has a clear bias for primary sources. The author favors NVIDIA PTX manuals, roofline analysis papers, and creator repositories over secondary explainers, and explicitly prescribes a “minimum mental model” in the “Start here” section before you dive into kernel tiling or speculative decoding.

Key highlights

• Covers the full vertical stack: GPU fundamentals, matrix multiplication and attention kernels, Triton and CUTLASS, inference scheduling, and distributed serving.
• Includes modern hardware targets: Hopper and Blackwell tuning guides, AMD ROCm, Google TPU, and AWS Trainium.
• Curated from original sources—links to FlashAttention-4, DeepGEMM, the Roofline model, and the canonical transformer inference arithmetic paper.
• Organized pedagogically: read from “one inference request” to “single GPU” to “distributed systems” rather than alphabetically.

Verdict

Grab this if you are an engineer who knows transformers work but wants to understand why they are slow and how to fix them. Skip it if you are looking for a framework or runnable benchmarks; this is a reading list, not a repo of code.