ildoonet/pytorch-gradual-warmup-lr
A PyTorch extension that gradually increases the learning rate during early training before switching to standard LR schedulers.
Not currently ranked — collecting fresh signals.
star history
Provides a GradualWarmupScheduler for PyTorch that incrementally raises the learning rate over a specified number of epochs at the start of training, as proposed in the ‘Accurate, Large Minibatch SGD’ paper. It can be chained with other PyTorch schedulers such as StepLR or cosine annealing. This technique helps stabilize early training when using large batch sizes by preventing large gradient updates initially.
Frequently asked
- What is ildoonet/pytorch-gradual-warmup-lr?
- A PyTorch extension that gradually increases the learning rate during early training before switching to standard LR schedulers.
- Is pytorch-gradual-warmup-lr open source?
- Yes — ildoonet/pytorch-gradual-warmup-lr is open source, released under the MIT license.
- What language is pytorch-gradual-warmup-lr written in?
- ildoonet/pytorch-gradual-warmup-lr is primarily written in Python.
- How popular is pytorch-gradual-warmup-lr?
- ildoonet/pytorch-gradual-warmup-lr has 987 stars on GitHub.
- Where can I find pytorch-gradual-warmup-lr?
- ildoonet/pytorch-gradual-warmup-lr is on GitHub at https://github.com/ildoonet/pytorch-gradual-warmup-lr.