allenai/longformer
Longformer is a transformer model variant designed to process long documents beyond the typical attention bottleneck of standard transformers.
Velocity · 7d
+1.0
★ / day
Trend
→steady
star history
Longformer extends the standard transformer architecture to handle long sequences by replacing full self-attention with a sliding window attention pattern combined with global attention. It provides both a base model and a LongformerEncoderDecoder (LED) variant for seq2seq tasks. The project includes pretrained model weights and integrates with HuggingFace Transformers for ease of use.