tatp22/linformer-pytorch
A PyTorch implementation of the Linformer transformer architecture that reduces self-attention complexity from O(n²) to O(n) using projected attention.
Not currently ranked — collecting fresh signals.
star history
This repository provides a complete encoder-decoder transformer with linear-time attention, based on the Linformer paper. It implements the projected attention mechanism that compresses the key and value matrices to achieve linear complexity while maintaining model performance. The implementation includes a language model class for easy experimentation with very long sequences (1000+ tokens) on standard hardware.
Frequently asked
- What is tatp22/linformer-pytorch?
- A PyTorch implementation of the Linformer transformer architecture that reduces self-attention complexity from O(n²) to O(n) using projected attention.
- Is linformer-pytorch open source?
- Yes — tatp22/linformer-pytorch is open source, released under the MIT license.
- What language is linformer-pytorch written in?
- tatp22/linformer-pytorch is primarily written in Python.
- How popular is linformer-pytorch?
- tatp22/linformer-pytorch has 424 stars on GitHub.
- Where can I find linformer-pytorch?
- tatp22/linformer-pytorch is on GitHub at https://github.com/tatp22/linformer-pytorch.