A Speech Toolkit That Cleans Audio Using Lips, Gestures, and Brainwaves

What it does ClearerVoice-Studio is a unified speech processing workbench from Alibaba’s Speech Lab. It wraps pre-trained models for denoising, separation, super-resolution, and target speaker extraction into one repository, adding training scripts and a built-in evaluation suite called SpeechScore so you can both run inference and benchmark the results.

The interesting bit The target speaker extraction module accepts conditioning signals that go far beyond a reference audio clip: it supports face recordings, body gestures, and even EEG brain signals. If you have ever wanted to isolate a speaker by staring at their lips—or by reading their brainwaves—this toolkit supports both.

Key highlights

• Tri-part architecture: ClearVoice for inference, Train for fine-tuning, and SpeechScore for metrics including PESQ, STOI, DNSMOS, SI-SDR, NISQA, and DISTILL_MOS.
• Pre-trained FRCRN and MossFormer models are already heavily battle-tested, with 3.0M and 2.5M uses respectively on ModelScope.
• Supports a wide codec menagerie—wav, mp3, flac, opus, aac, and others—via ffmpeg, in both mono and stereo at 16-bit or 32-bit precision.
• Recent updates added a NumPy array interface for pipeline embedding and PyPI distribution as clearvoice.
• Training scripts cover everything from 48 kHz super-resolution to audio-visual extraction conditioned on lip movement or body gestures.

Caveats

• HuggingFace demos carry tighter GPU limits than ModelScope, so heavy experimentation may run smoother on ModelScope.
• The project leans on modified third-party code from SpeechBrain, ESPnet, and TalkNet-ASD; much of the value is in integration and model packaging rather than wholly novel architectures.

Verdict Worth a look if you need a one-stop shop for speech enhancement research or product prototyping. Pure application developers who just need a black-box API should probably stick to the hosted demos instead.