OpenMoE is a family of open-sourced Mixture-of-Experts (MoE) Large Language Models. The project releases model weights, training data, strategies, and model architecture to the community. It includes OpenMoE-base and 8B variants, with explorations toward 34B scale models. The team provides training code and checkpoints along with a research paper analyzing routing behavior in MoE architectures.