The 800K-follower dubbing rig you can self-host

What it does YouDub WebUI ingests a YouTube or Bilibili video and outputs a fully dubbed edition in a target language. It orchestrates Demucs for vocal isolation, Whisper for transcription, an OpenAI-compatible API for translation, VoxCPM for voice-cloned synthesis, and FFmpeg for remixing and subtitle hardcoding. The author uses this exact stack to produce content for an 800,000-follower Bilibili channel, so the tool is shaped by real creator workflows rather than demo aesthetics.

The interesting bit Most automation projects treat audio separation as an afterthought; here it is a core stage, ensuring background music and sound effects survive under the new cloned voice. The stack is also deliberately monolithic—FastAPI backend, Next.js frontend, SQLite—so developers can trace a single video’s journey without debugging a mesh of microservices.

Key highlights

• Proven at scale: the author’s Bilibili channel, with 80万+ followers and 20,000+ videos, is entirely dubbed with this tool.
• End-to-end flow: handles download, vocal isolation, ASR, translation, voice-cloned TTS, remixing, and subtitle burning in one pass.
• Cross-platform sources: mature support for YouTube English-to-Chinese and experimental Bilibili Chinese-to-English.
• Local-first inference: Whisper, VoxCPM, and Demucs run on your own hardware, though translation still calls out to an external LLM API.
• Modest architecture: a single Python backend and TypeScript frontend, designed to be understood and modified rather than worshipped.

Caveats

• GPU is effectively mandatory: the README notes that CPU-only operation makes transcription, separation, and TTS “very slow,” and Apple Silicon mps forces Whisper to fall back to CPU due to unsupported float64 DTW alignment.
• Translation requires an external brain: you must supply an OpenAI-compatible Chat Completions endpoint and API key; the rest can run offline, but this step cannot.
• YouTube access needs grooming: fetching videos requires a local proxy and Netscape-format cookies, adding operational friction depending on your network region.

Verdict A strong fit for creators or small teams with a CUDA machine who want to own their localization pipeline end-to-end. If you prefer managed SaaS and zero GPU maintenance, look elsewhere.