A 7B audio model that listens, reasons, and talks back

What it does Kimi-Audio is a 7-billion-parameter foundation model built on a Qwen 2.5 7B backbone that ingests speech, music, and environmental sound. It handles automatic speech recognition, audio question answering, captioning, and end-to-end spoken conversation, generating both text replies and synthesized audio through parallel output heads. The repository contains inference code, pretrained and instruction-tuned weights, and a standalone evaluation toolkit.

The interesting bit The architecture treats audio as two parallel inputs: continuous acoustic vectors from a Whisper encoder and discrete semantic tokens at 12.5 Hz. On the output side, a flow-matching detokenizer paired with a BigVGAN vocoder translates semantic tokens back into waveforms in streaming chunks, giving the model a native voice with low latency rather than bolting on a separate TTS system.

Key highlights

• Pre-trained on over 13 million hours of diverse audio and text data.
• Claims state-of-the-art word-error rates on LibriSpeech, AISHELL-1, and WenetSpeech per the included benchmark tables.
• Supports multiturn audio conversations where the model accepts spoken prompts and replies with both speech and text.
• Ships with Kimi-Audio-Evalkit, a separate repository for reproducing the reported benchmarks and baselines.
• Includes a finetuning example for adapting the base checkpoint to downstream tasks.

Verdict Worth a look if you need a single open-weights model that both understands and generates audio. Skip it if you are looking for a lightweight edge solution; a 7-billion-parameter transformer with flow-matching detokenization is not a Raspberry Pi project.