← all repositories
amitshekhariitbhu/llm-inference-engineering

The reading list that explains why your LLM is slow

A curated, step-by-step blog series covering everything from KV cache and PagedAttention to speculative decoding and prefill-decode disaggregation.

★524 stars Markdown LearningInference · Serving
Too new to rank. Velocity needs a few days of history.
not enough history yet
star history

What it does This is not code — it’s a structured curriculum. Amit Shekhar (founder of Outcome School) has organized a series of linked blog posts into a progression: start with the autoregressive generation loop, then memory and attention optimizations, then throughput and latency tricks. Each README entry is a table of contents for one deep-dive article hosted on outcomeschool.com.

The interesting bit The ordering is the value. Instead of a pile of disconnected explainers, it builds a dependency chain: prefill vs decode → KV cache → why the cache explodes → PagedAttention and Flash Attention → continuous batching → speculative decoding and its descendants (Medusa, EAGLE, n-gram). By the end, terms like “prefill-decode disaggregation” stop sounding like vendor slides.

Key highlights

  • Covers the full inference stack: memory (KV cache compression, GQA, VRAM math), attention (PagedAttention, Flash Attention 1–3), and latency (continuous batching, speculative decoding variants)
  • Includes the metrics that actually matter in production: TTFT, TPOT, throughput, end-to-end latency
  • Each topic gets the “why is this slow” treatment before the fix, with worked numeric examples
  • The series is explicitly growing — more posts and videos are planned

Caveats

  • It’s a link hub, not a repo you run; all substance lives on the external blog
  • No benchmarks or code live in the repo itself

Verdict Great bookmark for engineers moving from “I call an API” to “I serve models” — or preparing for inference-focused interviews. Skip it if you already live inside vLLM internals.

Frequently asked

What is amitshekhariitbhu/llm-inference-engineering?
A curated, step-by-step blog series covering everything from KV cache and PagedAttention to speculative decoding and prefill-decode disaggregation.
Is llm-inference-engineering open source?
Yes — amitshekhariitbhu/llm-inference-engineering is open source, released under the Apache-2.0 license.
What language is llm-inference-engineering written in?
amitshekhariitbhu/llm-inference-engineering is primarily written in Markdown.
How popular is llm-inference-engineering?
amitshekhariitbhu/llm-inference-engineering has 524 stars on GitHub.
Where can I find llm-inference-engineering?
amitshekhariitbhu/llm-inference-engineering is on GitHub at https://github.com/amitshekhariitbhu/llm-inference-engineering.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.