This repository implements the Muon optimizer originally described in public posts. It is specifically designed to optimize hidden weights in neural networks while using AdamW for embeddings, classifier heads, and other parameters. The implementation provides a MuonWithAuxAdam class that allows mixing both optimizers in a single training run with separate parameter groups.