← all repositories

lilianweng/transformer-tensorflow

A TensorFlow implementation of the original Transformer model for sequence-to-sequence tasks like machine translation.

482 stars Python Language ModelsML Frameworks
transformer-tensorflow
Velocity · 7d
+0.2
★ / day
Trend
steady
star history

This repository provides a complete implementation of the Transformer architecture in TensorFlow, including encoder/decoder layers, multi-head self-attention mechanisms, and positional encoding. It includes training and evaluation scripts for machine translation tasks on standard benchmarks like WMT14 and IWSLT15. The project implements the core components — attention, feed-forward layers, residual connections, and label smoothing — that form the backbone of modern large language models.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.