KellerJordan/Muon
Muon is a custom optimizer for training the hidden weights of neural networks, designed to work alongside AdamW.
Not currently ranked — collecting fresh signals.
star history
This repository implements the Muon optimizer originally described in public posts. It is specifically designed to optimize hidden weights in neural networks while using AdamW for embeddings, classifier heads, and other parameters. The implementation provides a MuonWithAuxAdam class that allows mixing both optimizers in a single training run with separate parameter groups.
Frequently asked
- What is KellerJordan/Muon?
- Muon is a custom optimizer for training the hidden weights of neural networks, designed to work alongside AdamW.
- Is Muon open source?
- Yes — KellerJordan/Muon is open source, released under the MIT license.
- What language is Muon written in?
- KellerJordan/Muon is primarily written in Python.
- How popular is Muon?
- KellerJordan/Muon has 2.7k stars on GitHub.
- Where can I find Muon?
- KellerJordan/Muon is on GitHub at https://github.com/KellerJordan/Muon.