The Pipeline That Refuses to Do Multi-Line Subtitles

What it does

VideoLingo ingests a video—often straight from YouTube via yt-dlp—and runs it through word-level transcription with WhisperX, a three-step translation and reflection stage, and optional dubbing. It outputs Netflix-style single-line subtitles paired with synthesized audio, all wrapped in a Streamlit interface that lets you pause, resume, or swap models mid-process. You can run the stack entirely offline with Ollama and Edge-TTS or hand off the heavy lifting to commercial APIs.

The interesting bit

The project treats subtitle formatting as a first-class problem, not an afterthought: it uses NLP segmentation and a custom terminology engine to prevent the usual machine-translation awkwardness, and it actively rejects multi-line subtitles as a matter of policy. That obsession with line breaks is oddly refreshing in a space that usually ignores readability.

Key highlights

• Enforces single-line subtitles only, aiming for Netflix-grade readability
• Three-step “Translate-Reflect-Adaptation” pipeline with AI-generated terminology glossaries
• Word-level alignment via WhisperX, with voice separation for noisy sources
• Supports multiple TTS backends including GPT-SoVITS, Azure, OpenAI, and custom integrations
• Task-level pause/resume controls and progress resumption via detailed logging

Caveats

• WhisperX struggles with background noise and can truncate subtitles ending in numbers or special characters
• Multilingual source videos will lose non-dominant languages during transcription
• Weaker LLMs may crash the pipeline with JSON parsing errors, requiring you to wipe the output folder and restart

Verdict

Content creators and localization teams drowning in manual subtitle work should take a look. If you need reliable multilingual source handling or flawless multi-character dubbing, this isn’t there yet.