tomaarsen/attention_sinks
A library that modifies pre-trained LLMs to generate fluent text indefinitely beyond their original training context using attention sink mechanisms.
Not currently ranked — collecting fresh signals.
star history
The library adapts existing transformer-based LLMs to use a sliding window attention variant that maintains the ability to produce coherent text over arbitrarily long sequences. It does not require retraining — modifications are applied post-hoc to the attention mechanism. The project provides benchmark code comparing perplexity across multiple model families including Llama-2, Falcon, Mistral, and GPT-J under long-context generation scenarios.
Frequently asked
- What is tomaarsen/attention_sinks?
- A library that modifies pre-trained LLMs to generate fluent text indefinitely beyond their original training context using attention sink mechanisms.
- Is attention_sinks open source?
- Yes — tomaarsen/attention_sinks is open source, released under the Apache-2.0 license.
- What language is attention_sinks written in?
- tomaarsen/attention_sinks is primarily written in Python.
- How popular is attention_sinks?
- tomaarsen/attention_sinks has 735 stars on GitHub.
- Where can I find attention_sinks?
- tomaarsen/attention_sinks is on GitHub at https://github.com/tomaarsen/attention_sinks.