facebookresearch/coconut
A research implementation for training large language models to perform chain-of-thought reasoning directly in a continuous latent space rather than discrete tokens.

The repository implements a method for training large language models to reason by operating in a continuous latent space instead of generating discrete tokens at each reasoning step. It builds on chain-of-thought prompting approaches but eliminates the need for explicit text-based intermediate steps. The code supports training on reasoning datasets like GSM8K with configurable model architectures, training modes (Coconut, CoT, no-thoughts), and standard logging via wandb.