A diffusion code model that actually releases its training recipe

What it does Open-dLLM is a fully open diffusion-based language model for code generation. It ships with the entire lifecycle: a pretraining pipeline on the FineCode dataset, evaluation suites for HumanEval and MBPP, inference scripts, and released checkpoints for its 0.5B-parameter Open-dCoder model. The project builds on Qwen2.5-Coder, adapting it into a masked diffusion framework rather than the usual autoregressive approach.

The interesting bit This is the first diffusion LLM release to ship the full stack—training code, data, evaluation, and weights—rather than the usual inference scripts and checkpoints. It also supports representation alignment, which the authors say can adapt existing autoregressive models into diffusion LMs with a 4× speedup without retraining from scratch.

Key highlights

• Full stack openness: training data, code, checkpoints, and evaluation metrics are all published, unlike competitors that often release only inference scripts.
• A 0.5B-parameter model, Open-dCoder, initialized from Qwen2.5-Coder and trained with a masked diffusion objective.
• Representation alignment support for converting autoregressive LMs to diffusion LMs.
• Evaluation covers both standard code generation (HumanEval, MBPP) and code infilling benchmarks.
• Checkpoints and datasets are hosted on Hugging Face.

Verdict Researchers who want to train or study diffusion language models from scratch finally have a complete, open stack. If you just need a drop-in code completion model, standard autoregressive tools remain the pragmatic choice.