FasterDecoding/Medusa
Framework that adds extra decoding heads to LLMs to predict multiple future tokens simultaneously and accelerate generation.
Not currently ranked — collecting fresh signals.
star history
Medusa accelerates LLM inference by appending additional decoding heads that predict multiple future tokens in parallel. Unlike speculative decoding, it does not require a separate draft model. The original model remains unchanged while only the new heads are fine-tuned. During generation, tree-based attention combines candidates from all heads, and an acceptance scheme selects the longest plausible prefix for continued decoding.
Frequently asked
- What is FasterDecoding/Medusa?
- Framework that adds extra decoding heads to LLMs to predict multiple future tokens simultaneously and accelerate generation.
- Is Medusa open source?
- Yes — FasterDecoding/Medusa is open source, released under the Apache-2.0 license.
- What language is Medusa written in?
- FasterDecoding/Medusa is primarily written in Jupyter Notebook.
- How popular is Medusa?
- FasterDecoding/Medusa has 2.8k stars on GitHub.
- Where can I find Medusa?
- FasterDecoding/Medusa is on GitHub at https://github.com/FasterDecoding/Medusa.