← all repositories

harvardnlp/annotated-transformer

An annotated Jupyter notebook implementation of the Transformer architecture from the seminal "Attention Is All You Need" paper.

7.3k stars Jupyter Notebook LearningLanguage Models
annotated-transformer
Velocity · 7d
+2.4
★ / day
Trend
steady
star history

This repository provides a line-by-line annotated implementation of the Transformer model architecture in Python using Jupyter notebooks. It serves as an educational resource explaining how self-attention mechanisms, encoder-decoder structures, and transformer components work in practice. The material is maintained using jupytext to sync a Python script with the notebook format for version control, and can be run locally or via Google Colab.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.