Real-time voice AI that treats pipelines as composable agents

What it does

Pipecat is an open-source Python framework for building real-time voice and multimodal conversational agents. It orchestrates speech recognition, text-to-speech, conversation handling, audio and video streams, and network transports such as WebSockets or WebRTC into modular pipelines. You can run a single agent locally or distribute a fleet of them across processes and machines.

The interesting bit

The framework treats every pipeline as an agent that can hand off to specialists, fan out in parallel, or run as sidecars over a shared bus—so you’re designing a conversation system, not just a chatbot. The maintainers also think in full-stack product terms: the project ships client SDKs spanning React to ESP32, plus a CLI, a real-time debugger, and a terminal dashboard.

Key highlights

• Pluggable by design: the README lists nearly twenty speech-to-text providers, and the framework advertises broad support for many AI services and tools.
• Real-time native: built for ultra-low latency interaction over WebRTC and WebSockets rather than retrofitted streaming.
• Multi-agent architecture: supports handoffs, parallel fan-out, and distributed sidecars without forcing you to rebuild the message bus.
• Broad client coverage: official SDKs for JavaScript, React, React Native, Swift, Kotlin, C++, and ESP32.
• Satellite tooling: includes Pipecat Flows for structured conversations, Whisker for real-time debugging, Tail for terminal dashboards, and a CLI for scaffolding and deployment.

Caveats

• The README’s service table is truncated, so the full depth of non-STT integrations is left unclear from the provided source.
• The ecosystem sprawls across many satellite repositories—flows, UI kit, CLI, debugger, skills—so expect to juggle multiple projects to assemble the full stack.

Verdict

Worth evaluating if you’re building production-grade voice or video agents and need orchestration rather than a thin API wrapper. Skip it if you want a drop-in chatbot widget without the systems-design overhead.