From BPE to Flash Attention: a curated LLM deep-dive

What it does This repository is a curated learning syllabus—not a codebase—maintained by Outcome School founder Amit Shekhar. It collects links to blog posts and videos that walk through Large Language Model internals step by step, starting with tokenization and Byte Pair Encoding, moving through the mathematics of attention, backpropagation, and cross-entropy loss, and continuing into production inference optimizations like KV Cache, Paged Attention, Flash Attention, and Speculative Decoding.

The interesting bit The material targets the gap between “I use transformers” and “I understand why the softmax scaling factor is √dₖ.” Each explainer promises explicit numeric examples—walking through matrices, variance proofs, and masking implementations—rather than leaving everything to intuition. The progression mirrors how practitioners actually need to think: from training math to GPU memory tiling and continuous batching.

Key highlights

• Foundational math: attention Q/K/V mechanics, √dₖ scaling derivations, causal masking, backpropagation, and cross-entropy loss with numeric examples
• Architecture deep-dives: full Transformer breakdown, Feed-Forward Networks, and Batch vs. Layer Normalization
• Inference engineering: KV Cache, Paged Attention, Flash Attention, Speculative Decoding, and Continuous Batching
• Maintained as a living document with new topics added over time
• Content is hosted externally (Outcome School blogs and YouTube)

Caveats

• The README is currently truncated mid-section, leaving the Continuous Batching entry incomplete
• This is strictly an index of external blog posts and videos, not a codebase with implementations to run

Verdict Worth bookmarking if you are self-studying LLM mechanics and want a structured path from tokenization to production inference. Skip it if you are looking for a reference implementation or a library to import.