An open-source Neuro-sama that runs offline on your desktop

What it does

Open-LLM-VTuber is essentially glue code that wires local or remote LLMs, speech recognition, and text-to-speech pipelines to a Live2D avatar frontend. You can talk to it hands-free, interrupt it mid-sentence without headphones, and let it “see” your screen or webcam feed. It runs as either a web interface or a desktop client with a transparent, click-through pet mode that floats over your workspace.

The interesting bit

The project treats the boring plumbing—swapping between Ollama, Whisper variants, MeloTTS, CosyVoice, and many other backends—as its core feature. That modularity lets you reshape the persona, voice, and appearance without delving into the code, which is rarer than it sounds in the AI companion space.

Key highlights

• Runs fully offline with local models, or mixes in cloud APIs for resource-intensive tasks
• Desktop pet mode with transparent background, global top-most, and mouse click-through so the avatar lingers over your workspace
• Voice interruption without headphones; the AI won’t hear its own voice
• Visual perception via camera, screenshots, or screen recording
• Broad backend swap-ability: LLMs (Ollama, vLLM, LM Studio, OpenAI-compatible APIs), ASR (Whisper.cpp, Faster-Whisper, FunASR), TTS (Bark, Edge TTS, GPT-SoVITS, Fish Audio)

Caveats

• Long-term memory is temporarily removed; only chat log persistence remains for now
• The project warns it is in early stages and actively under development, with a v2.0 rewrite currently in planning
• Remote access requires HTTPS setup because browser microphone APIs demand a secure context

Verdict

Worth a look if you want a hackable, offline-first AI companion with a face. Skip it if you need production stability or would rather not edit a conf.yaml to swap backends.