Is Ranger-Deep-Learning-Optimizer open source?

Yes — lessw2020/Ranger-Deep-Learning-Optimizer is open source, released under the Apache-2.0 license.

What language is Ranger-Deep-Learning-Optimizer written in?

lessw2020/Ranger-Deep-Learning-Optimizer is primarily written in Python.

How popular is Ranger-Deep-Learning-Optimizer?

lessw2020/Ranger-Deep-Learning-Optimizer has 1.2k stars on GitHub.

Where can I find Ranger-Deep-Learning-Optimizer?

lessw2020/Ranger-Deep-Learning-Optimizer is on GitHub at https://github.com/lessw2020/Ranger-Deep-Learning-Optimizer.

← all repositories

lessw2020/Ranger-Deep-Learning-Optimizer

Three Gradient Papers Stuffed Into One Optimizer

Ranger bolts RAdam, LookAhead, and gradient centralization into a single PyTorch optimizer so you don't have to juggle three separate papers.

★1.2k stars Python ML Frameworks

View on GitHub ↗

Not currently ranked — collecting fresh signals.

star history

What it does

Ranger is a PyTorch optimizer that fuses three existing techniques—RAdam (rectified Adam), LookAhead, and Gradient Centralization—into a single training step. It applies gradient centralization by default to convolutional and fully-connected layers, and exposes toggles like use_gc if you want to experiment. The codebase also ships sibling variants named RangerVA and RangerQH with different parameter sets.

The interesting bit

The author treats optimizer design like stacking LEGO bricks rather than inventing new math, and the README is unusually honest about training dynamics: Ranger apparently “hovers” near the optimal loss valley unless you drop the learning rate for the final quarter of training. That specific recipe—75 % flat followed by a cosine descent—is presented as load-bearing, not optional.

Key highlights

Reportedly beat previous AdamW scores across 12 FastAI leaderboard categories.
Gradient Centralization (now GC2) is on by default for conv and fc layers, but can be disabled at initialization.
Supports differential learning rates across parameter groups.
Includes fixes for a memory-leak edge case when saving and reloading model weights.
The author recommends pairing it with the Mish activation function and a beta1 of 0.95 rather than the usual 0.90.

Caveats

The README itself points to Ranger21 as the newer beta version with “a host of new improvements,” so this repo appears to be in maintenance mode.
The latest changes (GC2, PyTorch 1.6 deprecation fixes) live in ranger2020.py and have not yet been merged into the main ranger.py file.

Verdict

Worth a look if you are training vision models in PyTorch and want an off-the-shelf drop-in replacement for AdamW with more knobs. Skip it if you are already using Ranger21 or need an optimizer with a clear, single theoretical lineage rather than a mash-up.

Frequently asked

What is lessw2020/Ranger-Deep-Learning-Optimizer?: Ranger bolts RAdam, LookAhead, and gradient centralization into a single PyTorch optimizer so you don't have to juggle three separate papers.
Is Ranger-Deep-Learning-Optimizer open source?: Yes — lessw2020/Ranger-Deep-Learning-Optimizer is open source, released under the Apache-2.0 license.
What language is Ranger-Deep-Learning-Optimizer written in?: lessw2020/Ranger-Deep-Learning-Optimizer is primarily written in Python.
How popular is Ranger-Deep-Learning-Optimizer?: lessw2020/Ranger-Deep-Learning-Optimizer has 1.2k stars on GitHub.
Where can I find Ranger-Deep-Learning-Optimizer?: lessw2020/Ranger-Deep-Learning-Optimizer is on GitHub at https://github.com/lessw2020/Ranger-Deep-Learning-Optimizer.