allenai/longformer
Longformer is a transformer model variant designed to process long documents beyond the typical attention bottleneck of standard transformers.
Not currently ranked — collecting fresh signals.
star history
Longformer extends the standard transformer architecture to handle long sequences by replacing full self-attention with a sliding window attention pattern combined with global attention. It provides both a base model and a LongformerEncoderDecoder (LED) variant for seq2seq tasks. The project includes pretrained model weights and integrates with HuggingFace Transformers for ease of use.
Frequently asked
- What is allenai/longformer?
- Longformer is a transformer model variant designed to process long documents beyond the typical attention bottleneck of standard transformers.
- Is longformer open source?
- Yes — allenai/longformer is open source, released under the Apache-2.0 license.
- What language is longformer written in?
- allenai/longformer is primarily written in Python.
- How popular is longformer?
- allenai/longformer has 2.2k stars on GitHub.
- Where can I find longformer?
- allenai/longformer is on GitHub at https://github.com/allenai/longformer.