scrya-com/rotorquant
KV cache quantization method for LLMs using block-diagonal rotations to compress transformer memory during inference.
Not currently ranked — collecting fresh signals.
star history
RotorQuant applies block-diagonal rotation matrices to compress key-value cache in transformer models, achieving 10.3x compression with improved perplexity and throughput versus TurboQuant. It reduces decode latency by 28% and prefill speed by 5.3x through planar/isolated rotation strategies that avoid the O(d log d) butterfly network overhead. Supports drop-in integration with llama.cpp for deployment.
Frequently asked
- What is scrya-com/rotorquant?
- KV cache quantization method for LLMs using block-diagonal rotations to compress transformer memory during inference.
- Is rotorquant open source?
- Yes — scrya-com/rotorquant is an open-source project tracked on heatdrop.
- What language is rotorquant written in?
- scrya-com/rotorquant is primarily written in Python.
- How popular is rotorquant?
- scrya-com/rotorquant has 1k stars on GitHub.
- Where can I find rotorquant?
- scrya-com/rotorquant is on GitHub at https://github.com/scrya-com/rotorquant.