A self-hosted AI receptionist that speaks SIP and LLM

What it does

Tel-Agent parks itself between a SIP trunk and an AI backend. When a call arrives, it checks caller ID against your routing rules and either forwards to a human, blocks the number, or hands it to a voice agent that streams speech recognition and synthesis in real time, interrupts mid-sentence, and invokes tools like calendar lookups or generic HTTP endpoints. Every conversation is recorded, transcribed, and dropped into a searchable archive. The same backend handles text channels—today web chat, SMS, email, WhatsApp, Telegram, Messenger, Instagram, Discord, and Slack—using credentials you own, not a shared third-party app.

The interesting bit

Instead of treating each messaging platform as a special snowflake, the project defines a single declarative channel contract; adding Signal, Teams, or IRC should be a new definition plus transport rather than a fork of the last adapter. The whole stack—FastAPI, Next.js, SQLite or PostgreSQL, Redis, and Caddy—ships as native Windows, macOS, and Linux packages or a Docker Compose fleet, built to run on your LAN with your own API keys.

Key highlights

• Self-hosted by design: runs on your hardware, on your network, with your own STT/LLM/TTS keys. v1 defaults to Deepgram, a cloud LLM, and ElevenLabs; local Ollama/Whisper/Piper support is planned for v1.1 and needs a GPU.
• PBX extension, not replacement: connects to 3CX, Asterisk, or FreePBX over SIP, and reaches automation platforms like n8n or Home Assistant through webhooks and an HTTP tool.
• Real-time voice targets under 800 ms from end-of-speech to first audio byte, streaming tokens while still generating the rest of the response.
• Alpha software with unsigned native installers; the Windows build starts the local service and opens the dashboard automatically.
• Dashboard covers routing rules, contacts, assistants, knowledge bases, conversation archives, and notification trays across all wired channels.

Caveats

• Alpha status with unsigned installers; expect an OS warning on first run.
• The phone channel needs SIP—analog lines require an external ATA—and the server must sit on the same LAN as the PBX with 5060/UDP access and an open RTP port range.
• v1 relies on cloud speech and inference providers; fully local voice requires a GPU and is deferred to v1.1.

Verdict

Organizations already running a PBX and looking to add an AI front desk without shipping call recordings to a SaaS should try the Docker image or unsigned installer. If you lack a SIP endpoint, a LAN-adjacent server, or patience for alpha software, this is not yet your receptionist.