A voice assistant that runs locally and speaks in your voice

What it does

This project is Python glue wiring OpenAI Whisper, a Langchain LLM backend, and ChatterBox TTS into a conversational loop. By default it keeps everything local via Ollama, though you can swap in MiniMax if you prefer cloud inference. It transcribes speech, generates a response, and speaks it back with optional voice cloning and tunable emotional expressiveness.

The interesting bit

The recent migration from Bark to ChatterBox brought a 0.5B-parameter TTS engine that supports voice cloning from a short audio clip and embeds a neural watermark in every generated file. A comically direct emotion-scoring function scans the LLM output for words like “amazing” or punctuation like “!” to decide how theatrical the reply should sound.

Key highlights

• Defaults to fully offline operation with Ollama, with an optional cloud LLM fallback via MiniMax
• ChatterBox TTS clones voices from 10–30 second samples and exposes exaggeration and cfg_weight controls for emotion and pacing
• Built-in neural watermarking in synthesized audio for authenticity verification
• Keyword-based dynamic emotion analysis adjusts expressiveness based on detected emotional cues in generated text
• Dependency setup prefers uv or pyproject.toml; the provided requirements.txt is explicitly flagged as potentially brittle across systems

Caveats

• The emotion analysis is currently a simple keyword scan rather than a nuanced sentiment model, so its judgment of tone is roughly as subtle as a thesaurus with a volume knob.
• The README warns that requirements.txt can fail on some systems because it was generated by uv pip freeze with exact pinned versions.

Verdict

Worth a look if you want a hackable, fully local voice assistant without subscription APIs. Skip it if you need production-grade emotion understanding or a polished web UI—this is a script you run in a terminal.