kimiyoung/transformer-xl
PyTorch and TensorFlow implementation of Transformer-XL, a recurrent transformer language model that extends context beyond fixed-length segments.
Not currently ranked — collecting fresh signals.
star history
This repository provides the official implementations of Transformer-XL in both PyTorch and TensorFlow, along with pretrained models. Transformer-XL introduced segment-level recurrence and relative positional encoding to handle longer sequences beyond fixed-length contexts. It achieved state-of-the-art results on multiple language modeling benchmarks including enwiki8, text8, One Billion Word, WikiText-103, and Penn Treebank.
Frequently asked
- What is kimiyoung/transformer-xl?
- PyTorch and TensorFlow implementation of Transformer-XL, a recurrent transformer language model that extends context beyond fixed-length segments.
- Is transformer-xl open source?
- Yes — kimiyoung/transformer-xl is open source, released under the Apache-2.0 license.
- What language is transformer-xl written in?
- kimiyoung/transformer-xl is primarily written in Python.
- How popular is transformer-xl?
- kimiyoung/transformer-xl has 3.7k stars on GitHub.
- Where can I find transformer-xl?
- kimiyoung/transformer-xl is on GitHub at https://github.com/kimiyoung/transformer-xl.