Stop Writing Intent Classifiers for Every New AI Agent

What it does Plano is a Rust-based proxy and data plane that sits between users and your AI agents. You declare agents and their capabilities in YAML, and it handles low-latency orchestration, LLM routing, safety guardrails, and OpenTelemetry tracing without touching your application code. Agents remain simple HTTP servers with an OpenAI-compatible chat endpoint; Plano takes care of the messy delivery infrastructure.

The interesting bit Instead of using heavy general-purpose models for routing, Plano uses its own lightweight LLMs—like a 4B-parameter orchestrator—to figure out which agent should handle a request. Because it is built on Envoy by the proxy’s core contributors, it treats agent traffic like modern service mesh traffic: filters, listeners, and declarative config rather than framework abstractions.

Key highlights

• Declare agents with natural-language descriptions in YAML; Plano routes requests automatically—no custom intent classifiers needed.
• Built-in filter chains for moderation, jailbreak protection, and memory hooks.
• Automatic OpenTelemetry traces and “Agentic Signals” for evaluation and continuous improvement.
• Model-agnostic LLM gateway that routes by name, alias, or preference, abstracting provider quirks.
• Runs as an out-of-process data plane, so agents can be written in any language or framework.

Caveats

• Free hosted routing LLMs (like the Plano-Orchestrator) run only from a US-central region; production scaling requires self-hosting or contacting the team for API keys.
• Some of the smarter routing features depend on Plano’s own model family, which is currently offered as a hosted service alongside the open-source proxy.

Verdict If you are shipping multi-agent systems to production and tired of reinventing routing, observability, and guardrails in Python middleware, Plano is worth a look. If you are still in the single-agent demo phase, it is probably overkill.