A traffic cop for your LLM API keys and endpoints

What it does

It is a Go-based transparent proxy that sits between your application and upstream AI providers. It preserves native API formats for OpenAI, Google Gemini, and Anthropic Claude while managing a high-performance pool of API keys. The proxy handles weighted load balancing across endpoints, automatic key rotation, and blacklisting of failed keys with recovery logic.

The interesting bit

The project treats API keys as a managed resource rather than static strings. It groups keys into pools, automatically pulls broken ones from rotation, and attempts recovery without human intervention. Configuration is split into two layers: static environment variables for infrastructure and dynamic database-backed settings that hot-reload without a restart, which is the kind of architectural split that keeps ops teams sane during incidents.

Key highlights

• Transparent proxy preserving native request/response formats for OpenAI, Gemini, and Claude.
• Intelligent key pool with group-based management, automatic rotation, and failure recovery.
• Dynamic configuration hot-reloads from the database without application restarts.
• Distributed leader-follower clustering with horizontal scaling for high-availability deployments.
• Vue 3 web management interface with real-time statistics, health checks, and request logging.

Verdict

Worth a look if you are running production traffic across multiple LLM providers and need key redundancy without writing your own middleware. Skip it if you are a solo developer making occasional API calls; your HTTP client is already enough.