laekov/fastmoe
A PyTorch extension providing fast and efficient Mixture of Experts (MoE) implementation for training large neural network models.
Not currently ranked — collecting fresh signals.
star history
FastMoE provides an easy-to-use system for supporting Mixture of Experts models in PyTorch. It contains customized C and Python operators for efficient distributed training of MoE layers. The project enables extending Transformer-based models with MoE architecture through a simple plugin interface, supporting features like distributed experts and NCCL-based communication.
Frequently asked
- What is laekov/fastmoe?
- A PyTorch extension providing fast and efficient Mixture of Experts (MoE) implementation for training large neural network models.
- Is fastmoe open source?
- Yes — laekov/fastmoe is open source, released under the Apache-2.0 license.
- What language is fastmoe written in?
- laekov/fastmoe is primarily written in Python.
- How popular is fastmoe?
- laekov/fastmoe has 1.9k stars on GitHub.
- Where can I find fastmoe?
- laekov/fastmoe is on GitHub at https://github.com/laekov/fastmoe.