A self-hosted LLM router that auto-picks the cheapest capable model

What it does OrcaRouter Lite is a FastAPI-based server that sits between your application and upstream LLM providers. You bring your own API keys, and it proxies OpenAI-compatible chat completion requests to the cheapest model that actually supports the capabilities you need—vision, tool calling, or JSON mode. If your local keychain doesn’t cover a model, it can treat the hosted OrcaRouter service as just another upstream provider. Everything runs locally by default: SQLite for state, an in-memory LRU for caching, and a single-page dashboard for analytics and key management.

The interesting bit The model="auto" feature is the main hook. Instead of hard-coding model names or building your own cost-optimization spaghetti, you send the same old OpenAI-shaped request and the server resolves the cheapest capable option from a catalog of 100+ models loaded at startup from LiteLLM’s community pricing database. It even exposes the resolved model in an x-orca-resolved-model header so you know exactly what you paid for. There is also a cross-provider prompt cache: deterministic requests are served from cache across any provider, not just Anthropic, which is the kind of boring infrastructure detail that actually saves money.

Key highlights

• model="auto" routes to the cheapest provider key you have that meets capability requirements (tools, vision, JSON mode).
• Hosted fallback: add an OrcaRouter cloud key and the self-hosted instance treats it as another upstream for models you don’t have keys for.
• 100+ model catalog with per-token pricing loaded from LiteLLM’s database at startup; no manual model list curation.
• Cross-provider exact-match prompt cache works across all configured providers, backed by Redis or an in-process LRU.
• Single-workspace by design: no multi-tenancy, RBAC, or billing complexity—just SQLite by default with optional Postgres and Redis.

Caveats

• Single-workspace only; if you need multi-tenancy, SSO, or audit logs, the README says to look at the hosted product or a forthcoming Teams edition.
• The project is explicitly scoped down: no Kubernetes deployment guides, no alert webhooks, no built-in billing or wallets.

Verdict Good for solo developers or small teams who want a self-hosted, drop-in OpenAI-compatible gateway with automatic cost optimization and a cloud safety net. Skip it if you need enterprise multi-tenancy or heavy governance features out of the box.