lucidrains/lion-pytorch
A PyTorch implementation of Lion, a neural network optimizer discovered via evolutionary search by Google Brain that claims improvements over AdamW for training deep learning models.
Not currently ranked — collecting fresh signals.
star history
This repository provides a PyTorch implementation of the Lion optimizer (Evolved Sign Momentum), an optimization algorithm for training neural networks that was automatically discovered through genetic program search by Google Brain. Lion claims to outperform AdamW on various benchmarks while being simpler in form. The implementation includes configurable hyperparameters for learning rate, weight decay, and beta values, along with guidance on adapting training schedules from AdamW to Lion.
Frequently asked
- What is lucidrains/lion-pytorch?
- A PyTorch implementation of Lion, a neural network optimizer discovered via evolutionary search by Google Brain that claims improvements over AdamW for training deep learning models.
- Is lion-pytorch open source?
- Yes — lucidrains/lion-pytorch is open source, released under the MIT license.
- What language is lion-pytorch written in?
- lucidrains/lion-pytorch is primarily written in Python.
- How popular is lion-pytorch?
- lucidrains/lion-pytorch has 2.2k stars on GitHub.
- Where can I find lion-pytorch?
- lucidrains/lion-pytorch is on GitHub at https://github.com/lucidrains/lion-pytorch.