Mixture-of-Agents (MoA) is an approach that leverages multiple LLMs in a layered architecture where each layer contains several LLM agents. These agents collaborate through multiple rounds of suggestion and refinement, allowing weaker models to contribute to stronger ensemble outputs. The framework achieves 65.1% on AlpacaEval 2.0 using only open-source models, outperforming GPT-4 Omni’s 57.5%.