bytedance/flux
A high-performance GPU kernel library enabling computation-communication overlap for distributed model training and inference.
Not currently ranked — collecting fresh signals.
star history
Flux provides optimized kernels for tensor and expert parallelism, targeting efficient training and inference of dense and MoE (Mixture of Experts) models on GPUs. It integrates with PyTorch and focuses on overlapping communication with computation to maximize GPU utilization during distributed training. The library is designed to support various parallelism strategies used in large-scale LLM training.
Frequently asked
- What is bytedance/flux?
- A high-performance GPU kernel library enabling computation-communication overlap for distributed model training and inference.
- Is flux open source?
- Yes — bytedance/flux is open source, released under the Apache-2.0 license.
- What language is flux written in?
- bytedance/flux is primarily written in C++.
- How popular is flux?
- bytedance/flux has 1.3k stars on GitHub.
- Where can I find flux?
- bytedance/flux is on GitHub at https://github.com/bytedance/flux.