Multi-agent LLMs that gossip in embeddings instead of English

What it does

LatentMAS is a multi-agent reasoning framework where agents communicate by swapping latent thoughts—hidden states and KV-cache entries—rather than generating long textual reasoning traces. It works with standard HuggingFace transformers and optionally vLLM backends, targeting math, science, coding, and medical QA benchmarks. The repository contains reproduction code for single-agent baselines, text-based multi-agent systems, and the latent-space method.

The interesting bit

The framework uses a training-free latent-space alignment that lets agents read each other’s “working memory” directly, skipping the usual ritual of writing intermediate thoughts in English. The authors report this cuts token usage by roughly 50–80% and wall-clock time by 3×–7× compared to standard text-based multi-agent or chain-of-thought baselines.

Key highlights

• Agents pass hidden states and KV-cache updates instead of text, operating in the model’s latent space.
• Compatible with any HuggingFace model; optional vLLM support via a hybrid two-GPU pipeline.
• Training-free alignment mechanism (latent_space_realign) treated as a task-specific hyperparameter.
• Reproduces experiments across 9 tasks including GSM8K, AIME, GPQA, MBPP+, and HumanEval+.
• Active ecosystem with community extensions for scientific discovery, heterogeneous agents, and kNN-based latent retrieval.

Caveats

• vLLM integration requires modifying internal vLLM backend code, which the README notes is not officially supported and may introduce minor numeric drift versus the HF backend.
• The latent_space_realign toggle is task-dependent, so you’ll need to experiment to find whether it helps for your specific model and benchmark.

Verdict

Worth a look if you’re building multi-agent LLM pipelines and drowning in token costs or latency. Skip it if you need fully supported, drop-in vLLM acceleration without backend hacks.