facebookresearch/large_concept_model
Meta's research on Large Concept Models, a 1.6B parameter sequence-to-sequence model that operates on semantic sentence representations in the SONAR embedding space.
Not currently ranked — collecting fresh signals.
star history
This repository provides official implementations of Large Concept Models, a language modeling approach that processes sentences as semantic concepts rather than individual tokens. The model uses the multilingual SONAR embedding space supporting 200+ languages and explores multiple generation approaches including MSE regression and diffusion-based methods. The work includes training recipes for 1.6B parameter models trained on approximately 1.3T tokens, built on top of fairseq2 and PyTorch.
Frequently asked
- What is facebookresearch/large_concept_model?
- Meta's research on Large Concept Models, a 1.6B parameter sequence-to-sequence model that operates on semantic sentence representations in the SONAR embedding space.
- Is large_concept_model open source?
- Yes — facebookresearch/large_concept_model is open source, released under the MIT license.
- What language is large_concept_model written in?
- facebookresearch/large_concept_model is primarily written in Python.
- How popular is large_concept_model?
- facebookresearch/large_concept_model has 2.4k stars on GitHub.
- Where can I find large_concept_model?
- facebookresearch/large_concept_model is on GitHub at https://github.com/facebookresearch/large_concept_model.