awslabs/awsome-distributed-ai
AWS reference architectures and test cases for distributed large model training using cloud HPC infrastructure.
Not currently ranked — collecting fresh signals.
star history
This repository provides reference architectures and benchmark scripts for training large models on AWS cloud infrastructure including SageMaker HyperPod, AWS ParallelCluster, PCS, Batch, and EKS. It includes cloud formation templates, AMI/container build scripts, and test cases covering various frameworks and parallel optimization strategies such as PyTorch DDP/FSDP, Megatron-LM, and NeMo. The repository also offers micro-benchmarks for NCCL, NCCOM, and NVSHMEM to measure performance and troubleshoot distributed training clusters.
Frequently asked
- What is awslabs/awsome-distributed-ai?
- AWS reference architectures and test cases for distributed large model training using cloud HPC infrastructure.
- Is awsome-distributed-ai open source?
- Yes — awslabs/awsome-distributed-ai is open source, released under the MIT-0 license.
- What language is awsome-distributed-ai written in?
- awslabs/awsome-distributed-ai is primarily written in Shell.
- How popular is awsome-distributed-ai?
- awslabs/awsome-distributed-ai has 452 stars on GitHub.
- Where can I find awsome-distributed-ai?
- awslabs/awsome-distributed-ai is on GitHub at https://github.com/awslabs/awsome-distributed-ai.