jiaweizzhao/GaLore
A PyTorch optimizer that reduces LLM training memory by projecting gradients into low-rank space.
Not currently ranked — collecting fresh signals.
star history
GaLore provides memory-efficient full-parameter learning for LLMs by projecting gradients into a low-rank subspace during training. It integrates with existing optimizers like AdamW, AdamW8bit, and Adafactor with minimal code changes. The method achieves comparable or better results than LoRA-style adapters while maintaining full-parameter learning capabilities, and has been extended with quantized variants like Q-GaLore.
Frequently asked
- What is jiaweizzhao/GaLore?
- A PyTorch optimizer that reduces LLM training memory by projecting gradients into low-rank space.
- Is GaLore open source?
- Yes — jiaweizzhao/GaLore is open source, released under the Apache-2.0 license.
- What language is GaLore written in?
- jiaweizzhao/GaLore is primarily written in Python.
- How popular is GaLore?
- jiaweizzhao/GaLore has 1.7k stars on GitHub.
- Where can I find GaLore?
- jiaweizzhao/GaLore is on GitHub at https://github.com/jiaweizzhao/GaLore.