Ship one LLM service instead of seventeen disconnected experiments

What it does

This is a structured 10-week curriculum—50 half-hour sessions—that walks you through building a single OpenAI-compatible inference service on a rented GPU. Each session adds one operational layer, from initial vLLM deployment to Prometheus/Grafana instrumentation, load testing past 1,000 concurrent requests, quantization, speculative decoding, and a cost-aware router. The goal is a reproducible benchmark and a serving stack you configured and tuned yourself, rather than a collection of disconnected experiments.

The interesting bit

The author rejects the usual scattershot tutorial format. Every session feeds the same living artifact, so you spend your time on inference economics and reading scheduler source rather than repeated setup. The ordering is deliberately opinionated: the roofline model comes first, instrumentation lands in week 3, and you read the vLLM block manager before you touch optimization knobs.

Key highlights

• One accumulating service: all fifty sessions feed the same endpoint, ending with a published benchmark with pinned versions and reproducible commands.
• Measurement-first discipline: dashboards and a load-test harness are built before any quantization or speculative decoding.
• Real hardware requirements: roughly one 24GB GPU rented by the half-hour, plus exactly two H100 sessions for the 1,000-concurrency test and disaggregation week.
• Engine comparison built in: vLLM (PagedAttention, V1 scheduler internals) and SGLang (RadixAttention, prefix reuse) are both deployed against the same model.
• Explicit scope boundary: edge runtimes like ONNX Runtime and TensorRT-LLM are optional and segregated to the final week because they share little operational surface with the datacenter stack.

Caveats

• You need to be comfortable with Python, transformers architecture, and the command line; prior serving or CUDA experience is not required, but there is no hand-holding on basics.
• It requires renting actual GPUs; the author budgets a 24GB card by the half-hour and two H100 sessions, so free-tier Colab will not carry you through the build days.
• Edge deployment is explicitly optional and cordoned off from the core datacenter curriculum.

Verdict

Worth your 25 hours if you want to move from “I read the vLLM paper” to “I can explain the scheduler from source and my Grafana dashboard proves it.” Skip it if you need a quick cheat-sheet or a drop-in Helm chart.