From LSTM to attention, one German sentence at a time

What it does

This repo is a curriculum of Jupyter notebooks that rebuild foundational sequence-to-sequence models in PyTorch, using German-to-English translation as the running exercise. It walks through encoder-decoder architectures built on LSTMs, GRUs, and attention mechanisms, leaning on torchtext, spaCy, and the evaluate library for data handling so the focus stays on the model mechanics. Each notebook is paired with a Colab link, so you can run the code without touching your local environment.

The interesting bit

Rather than dropping a full transformer on you and walking away, the tutorials treat seq2seq as a historical progression: you start with the information-compression headache of vanilla encoder-decoders, swap in GRUs, then add attention, experiencing exactly why each paper mattered. It is pedagogical glue code, but the glue is the point.

Key highlights

• Walks through implementations of multi-layer LSTM seq2seq, GRU encoder-decoder, and attention-based alignment.
• Uses standard PyTorch ecosystem tools (torchtext, spaCy, evaluate) rather than hiding the data pipeline.
• Each notebook includes an “Open in Colab” badge for immediate execution.
• Legacy versions of the tutorials are preserved for older torchtext APIs.
• The author explicitly solicits corrections and negative feedback via issues.

Caveats

• Previous tutorial versions already live in a legacy directory because torchtext breaking changes killed them; check your library versions before running locally.

Verdict

Grab this if you want to understand why attention won by actually feeling the pain of the architectures that preceded it. Skip it if you need a production translation pipeline or a reusable library; these are deliberately self-contained lesson plans.