The reading list that explains why your LLM is slow
A curated, step-by-step blog series covering everything from KV cache and PagedAttention to speculative decoding and prefill-decode disaggregation.
What it does This is not code — it’s a structured curriculum. Amit Shekhar (founder of Outcome School) has organized a series of linked blog posts into a progression: start with the autoregressive generation loop, then memory and attention optimizations, then throughput and latency tricks. Each README entry is a table of contents for one deep-dive article hosted on outcomeschool.com.
The interesting bit The ordering is the value. Instead of a pile of disconnected explainers, it builds a dependency chain: prefill vs decode → KV cache → why the cache explodes → PagedAttention and Flash Attention → continuous batching → speculative decoding and its descendants (Medusa, EAGLE, n-gram). By the end, terms like “prefill-decode disaggregation” stop sounding like vendor slides.
Key highlights
- Covers the full inference stack: memory (KV cache compression, GQA, VRAM math), attention (PagedAttention, Flash Attention 1–3), and latency (continuous batching, speculative decoding variants)
- Includes the metrics that actually matter in production: TTFT, TPOT, throughput, end-to-end latency
- Each topic gets the “why is this slow” treatment before the fix, with worked numeric examples
- The series is explicitly growing — more posts and videos are planned
Caveats
- It’s a link hub, not a repo you run; all substance lives on the external blog
- No benchmarks or code live in the repo itself
Verdict Great bookmark for engineers moving from “I call an API” to “I serve models” — or preparing for inference-focused interviews. Skip it if you already live inside vLLM internals.
Frequently asked
- What is amitshekhariitbhu/llm-inference-engineering?
- A curated, step-by-step blog series covering everything from KV cache and PagedAttention to speculative decoding and prefill-decode disaggregation.
- Is llm-inference-engineering open source?
- Yes — amitshekhariitbhu/llm-inference-engineering is open source, released under the Apache-2.0 license.
- What language is llm-inference-engineering written in?
- amitshekhariitbhu/llm-inference-engineering is primarily written in Markdown.
- How popular is llm-inference-engineering?
- amitshekhariitbhu/llm-inference-engineering has 524 stars on GitHub.
- Where can I find llm-inference-engineering?
- amitshekhariitbhu/llm-inference-engineering is on GitHub at https://github.com/amitshekhariitbhu/llm-inference-engineering.