Build voice AI toys on ESP32 without drowning in glue code

What it does

ElatoAI is a three-layer stack for building voice-controlled IoT devices. An ESP32-S3 client built in the Arduino framework captures audio and streams it over secure WebSockets to either Deno Edge or Cloudflare Workers, which then forward the audio to providers like OpenAI, Gemini, ElevenLabs, or Hume AI. A Next.js webapp on Vercel handles device registration, conversation history, and over-the-air firmware updates, effectively giving you a SaaS backend for a talking plushie or companion device.

The interesting bit

The ESP32 client does not need PSRAM—a notable constraint relaxation for cheap boards—and the project ships with Opus compression at 12 kbps to keep bandwidth low. The edge layer acts as a protocol translator: your toy speaks WebSocket, the cloud speaks proprietary realtime APIs, and the middleman keeps secrets off the device with optional AES-256 key encryption handled server-side.

Key highlights

• Supports five realtime voice APIs out of the box: OpenAI Realtime, Gemini Live, xAI Grok Voice, ElevenLabs Conversational AI, and Hume EVI-4
• Cloudflare Workers path can route to additional LLM, TTS, and STT models via Workers AI (you bring the API key)
• Firmware includes a WiFi captive portal, button/touch wake, volume and pitch control from the webapp, and OTA updates
• Global round-trip latency under 2–3 seconds; tested uninterrupted sessions up to 17 minutes
• Supabase handles auth, device management, and Postgres RLS for conversation transcripts

Caveats

• 2–3 second cold start when the ESP32 first connects to the edge server
• No speech interruption detection on the ESP32 hardware side
• Edge server stops when wall clock time is exceeded

Verdict

If you want to prototype a voice-enabled IoT gadget without building firmware, audio codecs, and cloud auth from scratch, this is a sensible shortcut. Pure software developers with no hardware interest will find the Arduino/PlatformIO client layer irrelevant.