LLM Chat, But Your Model Wears a VRM and Dances

What it does

Lobe Vidol is a Next.js platform that turns LLM conversations into theatrical performances. You pick a character from a marketplace—or upload your own VRM model—choose an AI backend from over a dozen providers, and chat via text or voice. In video chat mode, the character appears on a 3D stage, strikes Mixamo poses, and can load MMD dance files while it responds.

The interesting bit

The project treats the language model as just another cast member. It bolts together a character marketplace with automated i18n, a dance marketplace for MMD resources, VRM model support, touch-response editing, and TTS/STT synthesis into a single PWA. That breadth of integration—chat infrastructure, 3D rendering pipeline, and content marketplace—is the whole point.

Key highlights

• Supports a wide roster of LLM providers (Anthropic Claude, Google Gemini, AWS Bedrock, Groq, DeepSeek, Qwen, and several others).
• Built-in character editor for VRM models with customizable touch responses and stage backgrounds.
• MMD parser integration for loading .vmd dance files and .pmx stages.
• TTS and STT via OpenAI Audio and Microsoft Edge Speech.
• Character and dance marketplaces with community submissions and automated internationalization.

Caveats

• The project is explicitly in early beta, so expect rough edges and shifting APIs.
• The README lists many model providers but does not clarify which features work offline or how self-hosting affects the 3D asset pipeline.

Verdict

Worth a look if you are building VTuber tools, AI companions, or any interface where an avatar needs to emote beyond text bubbles. Skip it if you just need a headless chat API client.