fixie-ai/ultravox
A multimodal LLM that extends open-weight models (Llama, Mistral, Gemma) with a projector enabling direct audio understanding for real-time voice AI.
Not currently ranked — collecting fresh signals.
star history
Ultravox is a speech-capable multimodal LLM that processes audio directly without a separate ASR stage, converting audio into the high-dimensional space used by the underlying language model. This direct coupling allows faster responses than cascading ASR + LLM systems. The model builds on research from AudioLM, SeamlessM4T, and similar works, and versions have been trained on Llama 3, Mistral, and Gemma architectures.
Frequently asked
- What is fixie-ai/ultravox?
- A multimodal LLM that extends open-weight models (Llama, Mistral, Gemma) with a projector enabling direct audio understanding for real-time voice AI.
- Is ultravox open source?
- Yes — fixie-ai/ultravox is open source, released under the MIT license.
- What language is ultravox written in?
- fixie-ai/ultravox is primarily written in Python.
- How popular is ultravox?
- fixie-ai/ultravox has 4.5k stars on GitHub.
- Where can I find ultravox?
- fixie-ai/ultravox is on GitHub at https://github.com/fixie-ai/ultravox.