← all repositories

facebookresearch/schedule_free

PyTorch library implementing schedule-free optimization algorithms from Facebook Research.

2.3k stars Python ML FrameworksLearning
schedule_free
Velocity · 7d
+2.9
★ / day
Trend
steady
star history

A PyTorch library implementing Schedule-Free optimization algorithms that eliminate the need for learning rate schedules and warmup phases during training. It provides drop-in replacements for standard optimizers like SGD and AdamW, replacing momentum with interpolation and averaging. The approach is derived from the paper ‘The Road Less Scheduled’ (arXiv:2405.15682).

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.