microsoft/mup
A PyTorch package implementing maximal update parametrization (μP) for stable hyperparameter scaling across neural network widths.
Not currently ranked — collecting fresh signals.
star history
The mup package provides tools for implementing μP in PyTorch models, a technique that stabilizes optimal hyperparameters across different model sizes. It enables reliable hyperparameter transfer from small models to large ones, reducing uncertainty when scaling up neural networks. The research focuses on large pretrained transformers but applies generally to deep learning model scaling.
Frequently asked
- What is microsoft/mup?
- A PyTorch package implementing maximal update parametrization (μP) for stable hyperparameter scaling across neural network widths.
- Is mup open source?
- Yes — microsoft/mup is open source, released under the MIT license.
- What language is mup written in?
- microsoft/mup is primarily written in Jupyter Notebook.
- How popular is mup?
- microsoft/mup has 1.7k stars on GitHub.
- Where can I find mup?
- microsoft/mup is on GitHub at https://github.com/microsoft/mup.