Your LLM, now with a 3D face and feelings

What it does

Amica is a TypeScript frontend for conversing with 3D characters in the browser or via a Tauri desktop build. It hooks your microphone up to Whisper for speech recognition, pipes the result to a swappable LLM backend, and animates a VRM avatar while synthesizing a spoken response through one of several TTS engines. It also supports vision via BakLLaVA, so the character can respond to what its camera sees.

The interesting bit

The project is essentially a thick layer of glue between proven pieces—three.js and @pixiv/three-vrm for the avatar, Transformers.js for in-browser inference, and a laundry list of local and remote backends from Ollama to ElevenLabs. The emotion engine is the unusual seasoning: it generates response text tagged with emotional expressions and drives the avatar’s demeanor, which is more than most chat interfaces bother with.

Key highlights

• Supports a wide range of chat backends: OpenAI, OpenRouter, Ollama, Llama.cpp, KoboldCpp, Window.ai, and others.
• Speech pipeline uses Whisper for recognition and Silero VAD for voice activity detection.
• TTS options span cloud APIs (ElevenLabs, OpenAI) to local models (Coqui, Speech T5, AllTalkTTS, RVC).
• Vision support through BakLLaVA lets the avatar process visual input.
• Runs as a web app or a Tauri-based desktop application; supports importing custom VRM models.

Caveats

• The README carries an explicit warning that the Windows installer may unintentionally delete other files during uninstallation if you do not create a new folder for it.
• Documentation is noted as incomplete: the authors state docs “will be further updated soon” for the latest 1.2 release.

Verdict

Worth a look if you want a voice-first frontend for local or remote LLMs with a visual presence. Skip it if you need a mature, battle-tested desktop installer or fully fleshed-out documentation today.