What it does no-magic is a collection of 48 single-file Python scripts that reimplement core AI algorithms from scratch—GPT, BERT, diffusion, LoRA, PPO, Flash Attention, speculative decoding, and others. Each script trains and runs inference with no external dependencies, no PyTorch, no TensorFlow. The stated goal is understanding, not replacement: to show what frameworks hide under their abstractions.

The interesting bit The project treats complexity as a documentation failure. A complete autoregressive GPT or vision transformer fits in one readable file, which is either humbling for the frameworks or damning for how we teach this stuff—possibly both.

Key highlights

• 48 algorithms across foundations (GPT, RNN, LSTM, BERT, ViT, diffusion, VAE, GAN), alignment (LoRA, QLoRA, DPO, PPO, GRPO), and systems (Flash Attention, KV-cache, PagedAttention, quantization, speculative decoding, SSMs)
• Zero dependencies; pure Python 3.10+
• Each script is runnable end-to-end with training and inference
• Comparison scripts included (e.g., attention vs. none, Adam vs. SGD, RNN vs. GRU vs. LSTM)
• MIT licensed

Caveats

• README truncates mid-sentence in the sources, so coverage of later sections (04 and beyond) is unclear
• “No dependencies” means no GPU acceleration, no autograd, and no production viability—these are pedagogical toys, not competitors to PyTorch
• The “48 algorithms” badge is present but unverified against the actual file count in the truncated README

Verdict Grab this if you’re tired of treating transformers as incantations and want to see the matrix multiplications bare. Skip it if you need working models or are already comfortable reading CUDA kernels.