lilianweng/transformer-tensorflow
A TensorFlow implementation of the original Transformer model for sequence-to-sequence tasks like machine translation.
Not currently ranked — collecting fresh signals.
star history
This repository provides a complete implementation of the Transformer architecture in TensorFlow, including encoder/decoder layers, multi-head self-attention mechanisms, and positional encoding. It includes training and evaluation scripts for machine translation tasks on standard benchmarks like WMT14 and IWSLT15. The project implements the core components — attention, feed-forward layers, residual connections, and label smoothing — that form the backbone of modern large language models.
Frequently asked
- What is lilianweng/transformer-tensorflow?
- A TensorFlow implementation of the original Transformer model for sequence-to-sequence tasks like machine translation.
- Is transformer-tensorflow open source?
- Yes — lilianweng/transformer-tensorflow is an open-source project tracked on heatdrop.
- What language is transformer-tensorflow written in?
- lilianweng/transformer-tensorflow is primarily written in Python.
- How popular is transformer-tensorflow?
- lilianweng/transformer-tensorflow has 483 stars on GitHub.
- Where can I find transformer-tensorflow?
- lilianweng/transformer-tensorflow is on GitHub at https://github.com/lilianweng/transformer-tensorflow.