Routing Agentic Workflows by Action, Not Turn

The Action Is the Atomic Unit

Most LLM routers treat the request as the boundary. A prompt arrives; a model is chosen; a response leaves. But agentic systems do not think in requests. As Databricks describes them, they run a continuous perceive-reason-act-learn loop, maintaining state across interactions and invoking tools until a goal is reached [3]. A single conversational turn can contain multiple tool calls, file edits, or reasoning steps. Routing per turn means paying frontier-model prices for a shell command that a small model could handle, or dumping a complex planning step into a lightweight model that hallucinates.

WorkWeave Router’s core semantic bet is that the atomic unit of routing should be the action, not the turn. The project maintains explicit canonical terminology—session, round, turn, action, step—and routes at the action boundary. This sounds like pedantry until you look at the token math. Exadel notes that a 10,000-token system prompt repeated across a 50-turn session consumes half a million input tokens before any output is generated, and that the price spread between frontier and efficient models can be sixty-fold [5]. In that environment, misrouting one action inside a turn is not a rounding error; it is a budget leak.

A Local Scorer in a Multi-API World

The router is a drop-in proxy that natively handles Anthropic Messages, OpenAI Chat Completions, and Gemini generateContent, including streaming, tool use, and vision. Open-weights models—DeepSeek, Qwen, Llama, Mistral—arrive via any OpenAI-compatible endpoint such as OpenRouter. Client tools point at a single local port and keep working. What happens inside is more interesting than typical API aggregation.

Instead of cloud-based classification or hand-written regex rules, the router runs a tiny on-box embedder: an in-process cluster scorer derived from the Avengers-Pro work. The embedding and scoring happen locally, which means the prompt itself never leaves the box for classification. Provider keys stay local, encrypted at rest in Postgres. This architecture directly addresses the privacy and latency concerns that make many teams reluctant to send every internal prompt to a third-party classifier. Industry benchmarks suggest embedding-based routing adds roughly five milliseconds of overhead, compared with fifty to a hundred milliseconds for a full ML classifier [8]. For agentic workflows that may issue dozens of calls in sequence, that latency profile matters.

The system also emits OTLP traces out of the box, with a local dashboard or export to Honeycomb, Datadog, or Grafana. Observability is not an afterthought; it is a necessary corrective, because the scorer is making autonomous decisions about where your money goes.

The Cost-Quality Chasm

The economic pressure driving this category is severe. Digital Applied notes that the 2026 price spread between efficient models like DeepSeek V4 and frontier offerings like GPT-5.5-pro is roughly one hundred to one [8]. Peer-reviewed work from ICLR 2025 (RouteLLM) reported eighty-five percent cost savings on MT Bench while retaining ninety-five percent of GPT-4 quality, needing frontier models for only fourteen percent of queries [8]. Enterprise vendors in the space claim forty to seventy percent savings [2, 11].

But cheap routing carries a well-documented risk: silent quality regression. If the router silently downgrades a complex reasoning task to a cut-rate model, the failure mode is not a 500 error; it is a plausible-looking wrong answer [8]. The research community has begun treating this as an explainability crisis. A 2026 Georgia Tech paper introduces Topaz, a framework that replaces silent model assignments with skill-based profiling and fully traceable routing algorithms so developers can audit why a particular model was chosen [6]. Topaz’s critique is that existing systems optimize performance while leaving trade-offs unrecorded, preventing developers from distinguishing intelligent efficiency from latent failure [6].

WorkWeave’s answer to this tension is partially technical and partially operational. Technically, the cluster scorer embeds the action and matches it against model capabilities, which is more structured than a vibes-based prompt but less formally interpretable than Topaz’s explicit skill taxonomy. Operationally, the router exposes raw routing decisions as cursor-paginated NDJSON via an analytics export endpoint, allowing teams to pull decision logs into their own warehouse and look for drift. It is not a full auditability framework, but it is a recognition that black-box routing will not survive enterprise scrutiny.

Wiring Into the Agentic Stack

The project’s recent attention spike comes from its installer. A Node package auto-wires Claude Code, OpenAI Codex, opencode, and pi to the hosted or self-hosted endpoint, patching config files without forking the tools. For Claude Code, it adds slash-command-like directives that let users override the scorer on the fly. For Codex, it preserves the native ChatGPT OAuth login while injecting model-pin skills that the router parses from leading-space prompt forms. For pi, it adds a persistent route-and-savings line and a terminal mascot named Wooly. Cursor support is labeled early beta.

This integration strategy reveals the target user: developers already running multiple agentic coding tools and tired of maintaining separate API keys and model configs for each one. The router becomes a single local control plane. Self-hosting runs the full stack—Postgres, the router, and an optional dashboard—on a laptop. A managed Weave-hosted deployment exists for teams that do not want to operate Postgres. Both paths keep provider keys under the user’s control; even the hosted path uses a bring-your-own-key model, with prompts flowing directly to the chosen provider rather than through Weave’s infrastructure.

A model-selection API lets users disable entire model families from the terminal or dashboard, preventing the scorer from reaching into expensive or undesirable providers. This is the other half of the control plane: the router decides, but the human sets the guardrails.

The Hidden Markov Sidecar

Beneath the default cluster scorer lies an optional, more exotic policy layer: a frozen Hidden Markov Model sidecar that runs as a companion container on port 8093. The HMM is opt-in and does not alter the default strategy, suggesting the project is experimenting with sequential state-aware routing—using the history of actions to inform the next model choice—without forcing users to bet on an unproven policy.

This is where the project’s research lineage surfaces again. An HMM policy implies that the router could eventually treat the agent’s trajectory as a stochastic sequence, transitioning between cheap mode and frontier mode based on latent task states rather than isolated action embeddings. For now, it remains a sidecar curiosity, but it signals that WorkWeave sees the future of routing as temporal, not just transactional.

Limits and the Road Ahead

WorkWeave Router enters a crowded field. Production-ready routing tools already include OpenRouter, LiteLLM, Portkey, NotDiamond, Vercel AI Gateway, and Azure AI Foundry’s Model Router [8]. Many of these offer broader provider support or deeper enterprise governance. WorkWeave’s differentiation is its tight focus on the agentic action boundary and its local, privacy-first scorer.

What remains unclear is how the cluster scorer generalizes across domains. The README cites Avengers-Pro but offers no reproduced benchmark suite or public evaluation harness. Teams must therefore treat cost savings as a hypothesis to validate against their own workloads, using the analytics export and dashboard to catch regressions. The Elastic License v2 keeps source code visible while restricting competitive use, a choice that may slow community adoption relative to fully open alternatives. And while the project exports raw routing decisions, it has not yet embraced the formal skill-taxonomy or budget-traceability frameworks that researchers like Topaz argue are necessary for trustworthy autonomous routing [6].

Still, the bet is coherent. As agentic systems proliferate, the unit of optimization shrinks from the conversation to the individual action. WorkWeave is building the network layer for that shift—one that runs on your box, keeps your keys local, and tries, at least, to send the hard problems to the expensive brain and the easy ones to the cheap muscle.