zyushun/Adam-mini
A memory-efficient Adam optimizer variant for training deep learning and large language models.
Not currently ranked — collecting fresh signals.
star history
Adam-mini is a PyTorch optimizer that reduces Adam’s memory footprint by 50% while maintaining or exceeding performance. It partitions parameters into blocks based on Hessian structure and assigns a single learning rate per block, eliminating over 99.9% of redundant lr resources in the v state. The implementation is drop-in compatible with AdamW hyperparameters and supports transformer-specific configurations including multi-head attention parameters.
Frequently asked
- What is zyushun/Adam-mini?
- A memory-efficient Adam optimizer variant for training deep learning and large language models.
- Is Adam-mini open source?
- Yes — zyushun/Adam-mini is an open-source project tracked on heatdrop.
- What language is Adam-mini written in?
- zyushun/Adam-mini is primarily written in Python.
- How popular is Adam-mini?
- zyushun/Adam-mini has 458 stars on GitHub.
- Where can I find Adam-mini?
- zyushun/Adam-mini is on GitHub at https://github.com/zyushun/Adam-mini.