thunlp/InfLLM
A training-free memory mechanism that enables pre-trained LLMs to process extremely long sequences without fine-tuning.
Not currently ranked — collecting fresh signals.
star history
InfLLM stores distant contexts in additional memory units and employs an efficient lookup mechanism to retrieve token-relevant units for attention computation. This allows LLMs pre-trained on shorter sequences to process long inputs while maintaining the ability to capture long-distance dependencies. The method uses faiss for retrieval and requires no training or fine-tuning, making it directly applicable to LLM-driven agents with lengthy streaming inputs.
Frequently asked
- What is thunlp/InfLLM?
- A training-free memory mechanism that enables pre-trained LLMs to process extremely long sequences without fine-tuning.
- Is InfLLM open source?
- Yes — thunlp/InfLLM is open source, released under the MIT license.
- What language is InfLLM written in?
- thunlp/InfLLM is primarily written in Python.
- How popular is InfLLM?
- thunlp/InfLLM has 406 stars on GitHub.
- Where can I find InfLLM?
- thunlp/InfLLM is on GitHub at https://github.com/thunlp/InfLLM.