A self-hosted switchboard for your scattered LLM keys

What it does

Octopus sits between your code and upstream LLM services, giving you a single point of contact for OpenAI, Anthropic, Gemini, and others. You configure provider channels and API keys through a web panel, then expose them as unified model groups that clients call instead of hitting each provider directly. The gateway handles routing, protocol translation, and keeps a running tally of token spend.

The interesting bit

The group abstraction lets you pool several providers offering the same model under one name, then load-balance across channels using round-robin, weighted, or failover strategies. Per-channel, it can also pick the endpoint with the lowest latency, and it auto-syncs model lists and per-token prices from an upstream registry while letting you override prices manually.

Key highlights

• Pools multiple providers and API keys into unified model groups, with per-channel latency-based endpoint selection.
• Translates between OpenAI Chat, OpenAI Responses, and Anthropic API formats on the fly.
• Auto-syncs model lists and pricing from models.dev, with user-defined prices taking priority.
• Batches request statistics and cost tracking in memory before flushing to the database.
• Ships as a single Go binary with an embedded web UI, supporting SQLite, MySQL, or PostgreSQL.

Caveats

• In-memory statistics are lost on forced termination; the README explicitly warns against sending SIGKILL.
• Default login credentials are admin/admin, and the project nags you to change them immediately.

Verdict

Ideal for individual developers or small teams who self-host and want to consolidate multiple LLM subscriptions behind one endpoint with basic cost accounting. Skip it if you need a managed, multi-tenant enterprise gateway or deep request-level observability.