Gloo is a collective communications library that provides essential primitives for distributed machine learning training, including barrier, broadcast, and allreduce operations. It abstracts transport layers for IP and InfiniBand (or RoCE), supports GPUDirect for cross-machine GPU-to-GPU memory transfers without host-device copies, and works with both system memory and NVIDIA GPU memory buffers. It is the primary distributed communication backend for PyTorch.