Few-shot voice cloning that needs just one minute of audio

What it does

GPT-SoVITS is a voice cloning and text-to-speech system built in Python. It synthesizes speech in a new voice from a five-second audio clip, or fine-tunes for better realism on roughly one minute of data. The whole pipeline is wrapped in a WebUI that bundles audio separation, automatic segmentation, ASR, and text labeling so beginners can prep datasets without switching tools.

The interesting bit

It handles cross-lingual synthesis natively—train on a voice in one language and generate speech in English, Japanese, Korean, Cantonese, or Chinese. The maintainers also publish unvarnished speed numbers: 0.028 real-time factor on a 4060 Ti, 0.014 on a 4090, and a comparatively sluggish 0.526 on an Apple M4 CPU.

Key highlights

• Zero-shot TTS from a 5-second sample; few-shot fine-tuning with about 1 minute of audio
• Integrated WebUI tools for vocal/accompaniment separation, training-set segmentation, and Chinese ASR
• Cross-lingual inference across five supported languages without retraining the speaker embedding
• Packaged Windows builds, Docker images, and Colab notebooks for cloud or local use
• Active pretrained model hub with separate weights for UVR5, ASR, and phoneme conversion

Caveats

• macOS GPU training produces “significantly lower quality” than CUDA, so the project currently defaults to CPU on Apple Silicon
• Several auxiliary models must be downloaded manually and dropped into specific directory trees unless the automated install script handles them
• Docker images trail the rapid commit pace; the README suggests building locally if you need the newest changes

Verdict

A solid choice for developers or creators who want custom voices without curating hours of studio audio. Less appealing if you are looking for a fully managed, API-only TTS service with no local PyTorch wrangling.