BlackSamorez/tensor_parallel
A library that automatically shards PyTorch models across multiple GPUs for parallel training and inference.
Not currently ranked — collecting fresh signals.
star history
This library provides tensor parallelism to split large PyTorch models across multiple GPUs, enabling execution of models too large for a single device. It works with standard HuggingFace transformers and requires only wrapping the model with tp.tensor_parallel. Supports both inference (model.generate) and training (backward pass) workloads with examples including FLAN-T5 fine-tuning and OPT model parallelism.
Frequently asked
- What is BlackSamorez/tensor_parallel?
- A library that automatically shards PyTorch models across multiple GPUs for parallel training and inference.
- Is tensor_parallel open source?
- Yes — BlackSamorez/tensor_parallel is open source, released under the MIT license.
- What language is tensor_parallel written in?
- BlackSamorez/tensor_parallel is primarily written in Python.
- How popular is tensor_parallel?
- BlackSamorez/tensor_parallel has 655 stars on GitHub.
- Where can I find tensor_parallel?
- BlackSamorez/tensor_parallel is on GitHub at https://github.com/BlackSamorez/tensor_parallel.