Cheap model first, strong model on bail: routing coding agents by escalation

What it does

Agent-as-a-Router (ACRouter) is a research implementation of a model router for coding tasks. Instead of picking one model for everything, it starts tasks on cheap models and escalates to stronger ones only when a verifier rejects the result — a performance-per-dollar tradeoff, not a fixed model choice. The repo ships the full reproducible pipeline, the CodeRouterBench dataset (9,999 in-distribution tasks and 176 OOD tasks, each scored across 8 models), reference outputs, and demos for wiring the router into real workflows.

The interesting bit

The escalation logic is exposed as a small library: you hand ACRouter a candidate model list, a cheap chain, an escalation target, and your own call_model and verify callbacks. That makes the paper’s idea portable — the router doesn’t care whether your backend is an OpenRouter API, a local Codex CLI, or Claude Code. There are also ready-to-adapt integrations for Claude Code Router (gateway-level) and cc-switch (proxy-level), so routing happens before the request ever reaches a provider.

Key highlights

• Offline reproduction needs no API keys or live model calls — replays run entirely on cached results.
• CodeRouterBench is published on Hugging Face, with an optional trained LoRA router adapter (acrouter-qwen35-08b-router-lora).
• Config-driven pipeline accepts your own tasks.jsonl + model_results.jsonl if you already have task/model scores.
• Demos cover both API-based solving (with verifier-gated retries) and routing prompts into local Codex/Claude Code/Opencode CLIs.
• Headline numbers from the README: OOD176 shows AvgPerf 73.30 at $Total 86.72, versus ID replay at AvgPerf 50.14 for $22.31.

Caveats

• The repo is a paper reproduction first and a product second — the “runtime integrations” are described as ready-to-adapt, not turnkey.
• Older OOD112/SWE-MiniSandbox assets are legacy supplementary data; the current public benchmark is OOD176 only.
• The README doesn’t explain how the router’s decision model itself is trained beyond pointing to the optional LoRA adapter, so the core method’s details live in the paper.

Verdict

Worth a look if you’re paying for coding agents and suspect not every task needs the flagship model — the verifier-gated escalation pattern is genuinely reusable. Skip it if you wanted a polished drop-in router; this is research code with demos bolted on, and the interesting decisions are documented in the arXiv paper, not the README.