PKU-YuanGroup/MoE-LLaVA
A multi-modal large language model that uses Mixture-of-Experts architecture to efficiently handle vision-language tasks.
Not currently ranked — collecting fresh signals.
star history
MoE-LLaVA is a vision-language model that applies Mixture-of-Experts techniques to improve efficiency and performance in handling multi-modal inputs. The project implements sparse activation mechanisms where only a subset of expert networks are engaged per forward pass, enabling larger model capacity without proportional compute cost. It provides training code, pre-trained checkpoints, and interactive demos via HuggingFace and Replicate.
Frequently asked
- What is PKU-YuanGroup/MoE-LLaVA?
- A multi-modal large language model that uses Mixture-of-Experts architecture to efficiently handle vision-language tasks.
- Is MoE-LLaVA open source?
- Yes — PKU-YuanGroup/MoE-LLaVA is open source, released under the Apache-2.0 license.
- What language is MoE-LLaVA written in?
- PKU-YuanGroup/MoE-LLaVA is primarily written in Python.
- How popular is MoE-LLaVA?
- PKU-YuanGroup/MoE-LLaVA has 2.3k stars on GitHub.
- Where can I find MoE-LLaVA?
- PKU-YuanGroup/MoE-LLaVA is on GitHub at https://github.com/PKU-YuanGroup/MoE-LLaVA.