FairScale extends PyTorch with state-of-the-art distributed training techniques. It provides composable modules and easy-to-use APIs for scaling neural network training with limited compute resources. The library focuses on performance optimization through techniques like model parallelism, gradient accumulation, and mixed precision training.