lucidrains/CoCa-pytorch
A PyTorch implementation of CoCa, a multimodal image-text foundation model combining contrastive and generative training objectives.
Not currently ranked — collecting fresh signals.
star history
This repository provides a complete implementation of the CoCa (Contrastive Captioner) architecture, which jointly trains an image encoder and text decoder using both contrastive and captioning losses. The model uses a vision transformer for image encoding and a PaLM-style decoder for text generation, enabling image-text understanding and captioning capabilities. It supports cross-attention layers for multimodal fusion and achieves state-of-the-art accuracy on ImageNet benchmarks.
Frequently asked
- What is lucidrains/CoCa-pytorch?
- A PyTorch implementation of CoCa, a multimodal image-text foundation model combining contrastive and generative training objectives.
- Is CoCa-pytorch open source?
- Yes — lucidrains/CoCa-pytorch is open source, released under the MIT license.
- What language is CoCa-pytorch written in?
- lucidrains/CoCa-pytorch is primarily written in Python.
- How popular is CoCa-pytorch?
- lucidrains/CoCa-pytorch has 1.2k stars on GitHub.
- Where can I find CoCa-pytorch?
- lucidrains/CoCa-pytorch is on GitHub at https://github.com/lucidrains/CoCa-pytorch.