NVIDIA/kvpress
A Python library implementing multiple KV cache compression methods to reduce LLM inference memory usage.
Not currently ranked — collecting fresh signals.
star history
kvpress provides “presses” that compress the key-value cache during the prefilling phase of LLM inference. It wraps Hugging Face Transformers with custom compression strategies, allowing deployment of long-context models like Llama 3.1-70B with significantly reduced memory requirements. The project includes benchmarking tools and integrates with existing transformer pipelines for easy adoption.
Frequently asked
- What is NVIDIA/kvpress?
- A Python library implementing multiple KV cache compression methods to reduce LLM inference memory usage.
- Is kvpress open source?
- Yes — NVIDIA/kvpress is open source, released under the Apache-2.0 license.
- What language is kvpress written in?
- NVIDIA/kvpress is primarily written in Python.
- How popular is kvpress?
- NVIDIA/kvpress has 1.1k stars on GitHub.
- Where can I find kvpress?
- NVIDIA/kvpress is on GitHub at https://github.com/NVIDIA/kvpress.