Your Agent Thinks It Talks to OpenAI. Switchyard Knows Better.

What it does

Switchyard is a Rust proxy and library that intercepts LLM requests from clients speaking OpenAI or Anthropic APIs, translates them into whatever format the target backend expects, and translates the response back. It effectively lets you point Claude Code or Codex at a local vLLM, NVIDIA NIM, or Ollama instance without the agent realizing it left home. The same machinery handles traffic splitting, A/B benchmarking, and operational metrics via Prometheus.

The interesting bit

The routing logic goes beyond simple round-robin: it includes an LLM-as-classifier strategy that reads a request to decide whether to spend money on a strong model or save cash with a weak one, plus a stage router that uses conversation signals like tool errors to route turns without burning an extra inference. You can also embed the routing engine as a library in your own Rust application, handing the actual HTTP calls back to your code so it doesn’t impose its own network stack.

Key highlights

• Translates between OpenAI Chat, Anthropic Messages, and OpenAI Responses formats bidirectionally.
• Supports random splitting, LLM classifier routing, escalation routing, and signal-driven stage routing.
• Exposes Prometheus metrics for requests, errors, latency, tokens, and routing overhead.
• Can run as a standalone proxy, a coding-agent launcher wrapper, or an embeddable Rust library (switchyard-libsy).
• Pre-alpha software; the API and algorithms are expected to change significantly before v1.0.

Caveats

• Explicitly marked experimental and not for production use; APIs and algorithms will shift before v1.0.
• The library crate (switchyard-libsy) delegates model calls back to the host application, so it is not a drop-in HTTP client—you must pair it with switchyard-llm-client or your own stack if you want it to actually fire requests.

Verdict

Worth a look if you’re building LLM gateways or trying to benchmark cheap against expensive models without forking your client code. Skip it if you need battle-tested production infrastructure today; the README warns it is pre-alpha and evolving rapidly.