Mandarin-first ASR with an LLM mode and a 1.1B efficient twin

What it does

FireRedASR is a family of speech-to-text models built for Mandarin, Chinese dialects, and English. It ships two variants: FireRedASR-LLM, an 8.3B-parameter encoder-adapter-LLM framework that leverages Qwen2-7B-Instruct, and FireRedASR-AED, a 1.1B-parameter attention-based encoder-decoder meant for faster inference and downstream speech-LLM use. The project also claims strong performance on sung lyrics, a niche most ASR releases ignore.

The interesting bit

The README posts benchmark tables showing both variants beating Seed-ASR, Qwen-Audio, SenseVoice-L, and Whisper-Large-v3 on standard Mandarin sets—sometimes by a factor of ten in character error rate. The LLM variant also edges previous SOTA on English LibriSpeech test-clean. It is unusual to see an open-source release explicitly optimize for both boardroom meetings and karaoke.

Key highlights

• Dual architectures: a heavy LLM-backed model for accuracy and a sub-2B AED model for efficiency.
• Claims top public benchmark results on Mandarin (AISHELL-1/2, WenetSpeech) and competitive English numbers.
• Evaluated on Chinese dialects via KeSpeech.
• Explicitly advertises singing lyrics recognition.
• The team has already released FireRedASR2S, an all-in-one successor adding VAD, language ID, and punctuation modules.

Caveats

• FireRedASR-LLM requires a separate Qwen2-7B-Instruct download and is capped at roughly 30 seconds of audio; behavior beyond that is unknown.
• FireRedASR-AED hallucinates after 60 seconds and crashes on positional encoding beyond 200 seconds.
• Batch beam search with the LLM variant demands utterances of similar length, or shorter clips may repeat.

Verdict

Worth testing if you need Mandarin or Chinese dialect transcription and have found general-purpose ASR lacking. Skip it if you need reliable long-form transcription out of the box, or if you want the newer all-in-one system—head to FireRedASR2S instead.