Java backend gives ESP32 voice assistants an enterprise dashboard

What it does

It’s a complete Java rewrite of the backend for the Xiaozhi ESP32 open-source voice assistant. The project ships a Vue.js management dashboard and splits runtime into two processes: xiaozhi-server handles REST APIs, user management, and OTA updates, while xiaozhi-dialogue manages WebSocket/MQTT audio streams and the AI inference pipeline. Both share MySQL and Redis, and the dialogue service can scale horizontally behind a load balancer.

The interesting bit

The dual-process architecture is the quietly clever part: by decoupling the admin dashboard from the real-time audio service, you can scale the chatty voice pipeline without dragging the management UI along for the ride. It also integrates a laundry list of Chinese and international AI services—OpenAI, Zhipu, iFlytek, Ollama, Dify, Coze—and supports MCP tool protocols, RAG knowledge bases, and IoT function calls, making it less a simple bridge and more a full integration layer.

Key highlights

• Dual-process architecture: xiaozhi-server for admin/OTA, xiaozhi-dialogue for real-time audio, sharing MySQL/Redis
• Horizontal scaling of dialogue nodes with auto-registration and OTA load balancing
• Pluggable LLM/STT/TTS providers including OpenAI, Zhipu, iFlytek, Ollama, Dify, Coze, and local engines like Vosk and sherpa-onnx
• Full-duplex voice pipeline with local/cloud STT, TTS, voice cloning, real-time interruption, and bidirectional streaming
• Vue.js dashboard for device fleets, role switching, dialogue logs, and firmware upgrades
• Benchmarked to 100 concurrent device connections on 8C/8G cloud hardware with ~1.8s end-to-end latency

Caveats

• Some features shown in the comparison table are locked behind a commercial version and not included in the open-source release
• Native ML models and libraries are excluded from the repository and must be fetched via external scripts before the first run
• The codebase is purpose-built for the Xiaozhi ESP32 firmware; it is not a drop-in backend for other hardware

Verdict

Grab it if you are running Xiaozhi ESP32 devices and need a Java-native, dashboard-driven backend that can handle a fleet. Pass if you are looking for a generic, hardware-agnostic voice assistant server or a lightweight Python alternative.