The repository implements a novel masked diffusion approach for training language models. Instead of traditional autoregressive training, it uses a substitution-based parameterization that simplifies the absorbing state diffusion loss into a mixture of masked language modeling losses. The model achieves state-of-the-art perplexity results and offers an alternative to standard language model training approaches using diffusion-based denoising.