tensorgi/TPA
A transformer-based foundation model architecture using Tensor Product Attention mechanisms for improved efficiency and reduced KV cache, published at NeurIPS 2025.
Not currently ranked — collecting fresh signals.
star history
TPA (Tensor Product Attention Transformer) is a state-of-the-art language model architecture that replaces standard attention with Tensor Product Attention to enhance performance and reduce memory footprint. The repository provides official implementations for data preparation, model pretraining, and evaluation across different model scales. It also includes FlashTPA optimized decoding and prefilling kernels for efficient inference.
Frequently asked
- What is tensorgi/TPA?
- A transformer-based foundation model architecture using Tensor Product Attention mechanisms for improved efficiency and reduced KV cache, published at NeurIPS 2025.
- Is TPA open source?
- Yes — tensorgi/TPA is open source, released under the MIT license.
- What language is TPA written in?
- tensorgi/TPA is primarily written in Python.
- How popular is TPA?
- tensorgi/TPA has 459 stars on GitHub.
- Where can I find TPA?
- tensorgi/TPA is on GitHub at https://github.com/tensorgi/TPA.