veScale is Byted’s internal PyTorch Distributed library designed for hyperscale distributed training of large language models and reinforcement learning systems. It provides components like RaggedShard DTensor for efficient tensor sharding across distributed compute resources. The project open-sources select portions of its infrastructure for community benefit.