The Transformer Paper, Annotated in Python

What it does This repository holds the source for The Annotated Transformer, a Harvard NLP blog post that provides an annotated implementation of the original Transformer paper. Explanations and code share the same page, turning dense research into a literate executable document. It functions as a line-by-line technical translation of one of NLP’s foundational architectures.

The interesting bit Rather than committing the Jupyter notebook directly, the project treats .ipynb files as build artifacts. The canonical source is a plain Python script synced via jupytext, which keeps git diffs clean and prevents cell outputs from bloating repository history. It is a quietly elegant fix for the version-control headaches that usually plague notebook-based projects.

Key highlights

• Interleaves paper annotations with runnable Python code in a single narrative
• Uses jupytext to maintain a .py source file that generates the notebook on demand
• Enforces PEP8 compliance automatically through GitHub Actions, black, and flake8
• Has accumulated over 7,300 stars, suggesting it is a standard educational reference
• Includes a Colab badge linking to a hosted runnable version

Verdict A strong reference if you are learning or teaching the original Transformer architecture from first principles. Less useful if you need a modern, production-scale training framework — this is pedagogy, not infrastructure.