Hivemind enables training large neural networks across hundreds of computers from different organizations without a central coordinator. It uses a Distributed Hash Table for peer-to-peer communication, implements fault-tolerant backpropagation, and supports decentralized Mixture-of-Experts for scaling models beyond single machine memory. The library targets academic and research use cases where participants contribute computational resources.