lucidrains/transfusion-pytorch
A PyTorch library implementing MetaAI's Transfusion, a multi-modal model that jointly performs next-token language prediction and image generation using flow matching.
Not currently ranked — collecting fresh signals.
star history
This library provides a PyTorch implementation of the Transfusion architecture, which unifies autoregressive language modeling with continuous diffusion-based generation in a single transformer model. It handles mixed sequences of text tokens and continuous modality representations (such as images encoded as latents), enabling training on interleaved text-image data. The implementation supports classifier-free guidance for improved generation quality and can be extended to arbitrary numbers of modalities.
Frequently asked
- What is lucidrains/transfusion-pytorch?
- A PyTorch library implementing MetaAI's Transfusion, a multi-modal model that jointly performs next-token language prediction and image generation using flow matching.
- Is transfusion-pytorch open source?
- Yes — lucidrains/transfusion-pytorch is open source, released under the MIT license.
- What language is transfusion-pytorch written in?
- lucidrains/transfusion-pytorch is primarily written in Python.
- How popular is transfusion-pytorch?
- lucidrains/transfusion-pytorch has 1.4k stars on GitHub.
- Where can I find transfusion-pytorch?
- lucidrains/transfusion-pytorch is on GitHub at https://github.com/lucidrains/transfusion-pytorch.