Windows desktop engine gives local LLMs a voice, memory, and anime body

What it does

Soul of Waifu is a thick Windows desktop client that wraps local LLM inference (via llama.cpp), speech synthesis, and Live2D/VRM avatar rendering into a single offline roleplay environment. You chat by text or voice, the avatar reacts with 28 context-driven emotions and lip-sync, and the app uses auto-summarization and vector RAG to keep long conversations from overflowing the context window. It also imports characters from third-party hubs like Character AI and Chub AI.

The interesting bit

The “Desktop Companion Mode” is the standout gimmick: the avatar breaks out of the application window and floats on your screen with a transparent background, turning the character into a persistent desktop pet that can interrupt and be interrupted mid-sentence during voice calls. It treats the AI less like a chatbot and more like a low-budget virtual YouTuber that lives inside your monitor bezels.

Key highlights

• Runs local GGUF models through an integrated llama.cpp backend with CUDA/Vulkan support, or switches to cloud APIs (OpenAI, Mistral, Character AI, OpenRouter) without external setup.
• Real-time full-duplex voice chat with interruption support and lip-sync for Live2D/VRM models.
• Desktop Companion Mode renders the avatar as an overlay outside the main window.
• Built-in character gateway for one-click imports from community libraries and third-party platforms.
• Auto-summarization and “Smart Vector Memory” (RAG) to manage long-term context across thousands of messages.

Caveats

• Windows-only; the README explicitly targets Windows 10/11 with .bat installers and .exe launches.
• The project is built in Python but shipped as a pre-packaged, “no coding required” Windows app, so developers looking for a clean, hackable library may find a thick, opaque binary instead.
• Local mode is free and private, but the README recommends 16GB+ RAM and a discrete GPU for reasonable performance.

Verdict

Worth a look if you want a self-contained, offline AI companion with actual avatar puppeteering rather than yet another web UI. Skip it if you are on Linux or macOS, or if you were hoping for a lightweight, headless Python library to build your own frontend.