Local ML Replaces Manga Text, Not Just Translates It

What it does

Koharu is a desktop application that feeds a manga page through a staged pipeline of local machine-learning models. It detects speech bubbles and text blocks, performs OCR, inpaints the original lettering to restore the background, translates the text with an LLM, and renders the result back onto the page with manga-specific typography. The app is built in Rust around Tauri, candle, and llama.cpp, and keeps all inference on your hardware.

The interesting bit

Instead of treating translation as a single generative prompt, Koharu decomposes the job into specialized vision models—one for bubble detection, another for OCR, a separate inpainting network to erase source text, and a font-analysis model to match style—then stitches the page back together with a custom text engine that handles vertical CJK, right-to-left scripts, and constrained speech-bubble layouts. It even exports layered PSDs so human editors can clean up the machine’s work.

Key highlights

• Runs vision models and LLMs locally; no cloud required for the core pipeline.
• Supports CUDA, Metal, Vulkan, and experimental ZLUDA, with CPU fallback when GPU acceleration is unavailable.
• Renders translated text with a dedicated manga lettering engine rather than generic OS text primitives.
• Offers headless mode, a local HTTP API, and a built-in MCP server for automation and agent integration.
• Exports editable Photoshop PSDs with translated text on separate layers.

Caveats

• Vulkan acceleration is limited to OCR and LLM inference; detection and inpainting still require CUDA, Metal, or the experimental ZLUDA layer.
• NVIDIA GPUs need compute capability 8.0 or higher for CUDA support.
• The full model stack is downloaded automatically on first use, so expect a sizable initial fetch.

Verdict

Worth a look if you translate manga or build local-first media-automation tools. Skip it if you just need a quick browser-based translation or lack the GPU memory to run a multi-model vision pipeline.