Seven bottlenecks that got us from LeNet to GPT-4o

What it does

This repository is a curated walk through deep learning history, from early feed-forward networks to GPT-4o. For each milestone, the author collects critical papers, adds personal notes and math intuition, and provides toy PyTorch implementations where relevant. The whole thing is structured around a single thesis: progress in AI is best understood as a series of hacks that raised the ceiling on seven limiting factors.

The interesting bit

Instead of treating AlexNet or BERT as isolated miracles, the project slots them into a constraints framework—data, parameters, optimization, architecture, compute, efficiency, and energy. It argues that once you see the field this way, the path from MNIST to massive language models looks less like magic and more like a predictable engineering campaign against hard physical and informational limits.

Key highlights

• Each major era includes links to foundational papers, the author’s notes, and small PyTorch reproductions.
• The overview is written to be readable without running any code; the repository is essentially a long-form essay with optional homework.
• The data constraint section explicitly credits ImageNet for enabling AlexNet and BERT for unlocking internet-scale pre-training.
• The author borrows the structural conceit of Will and Ariel Durant’s The Lessons of History, treating deep learning as a civilization-scale story rather than a stack of disconnected papers.

Verdict

Worth a read if you want a coherent, opinionated narrative that connects decades of papers into a single causal chain. Give it a pass if you need a rigorous, citation-heavy survey or a hands-on coding curriculum; the value here is in the framing, not the volume of code.