From Word2Vec to BERT, each in under 100 lines of PyTorch

What it does

nlp-tutorial is a set of Jupyter notebooks that rebuild foundational NLP models in PyTorch, from 2003’s NNLM up through 2018’s BERT. Each implementation is deliberately compact—most clock in under 100 lines of actual code, comments and whitespace excluded. Every notebook is paired with its seminal paper and a Colab link, so you can read the math, then watch it run.

The interesting bit

The project treats terseness as pedagogy: by refusing framework abstractions and production boilerplate, it forces the model architecture itself to become the documentation. It is essentially a reading list where the text is executable PyTorch.

Key highlights

• Covers a full arc of deep NLP: embeddings, CNNs, RNNs, attention mechanisms, and Transformers.
• Each entry links directly to the original research paper and a runnable Google Colab notebook.
• Old TensorFlow v1 code has been archived; the active curriculum targets PyTorch 1.0+ for readability.
• The README claims every model stays under 100 lines of code, excluding comments and blank lines.

Caveats

• The curriculum is labeled “(Example Purpose),” so these are teaching sketches rather than battle-hardened training pipelines.
• The syllabus stops at BERT (2018); more recent architectures are absent.

Verdict

Reach for this if you are a student or researcher who wants to see the mechanical guts of attention or BERT without framework indirection. Look elsewhere if you need production-grade libraries, pre-trained checkpoints, or coverage of post-2018 NLP.