kuleshov-group/mdlm
A research project introducing MDLM, a masked discrete diffusion language model with substitution-based parameterization published at NeurIPS 2024.
Not currently ranked — collecting fresh signals.
star history
The repository implements a novel masked diffusion approach for training language models. Instead of traditional autoregressive training, it uses a substitution-based parameterization that simplifies the absorbing state diffusion loss into a mixture of masked language modeling losses. The model achieves state-of-the-art perplexity results and offers an alternative to standard language model training approaches using diffusion-based denoising.
Frequently asked
- What is kuleshov-group/mdlm?
- A research project introducing MDLM, a masked discrete diffusion language model with substitution-based parameterization published at NeurIPS 2024.
- Is mdlm open source?
- Yes — kuleshov-group/mdlm is open source, released under the Apache-2.0 license.
- What language is mdlm written in?
- kuleshov-group/mdlm is primarily written in Python.
- How popular is mdlm?
- kuleshov-group/mdlm has 701 stars on GitHub.
- Where can I find mdlm?
- kuleshov-group/mdlm is on GitHub at https://github.com/kuleshov-group/mdlm.