Go gateway for metering LLM access before the bill arrives

What it does

BricksLLM sits between your application and LLM providers, acting as a reverse proxy with a built-in billing department. You create API keys with granular rate and spend limits, then route OpenAI, Anthropic, Azure OpenAI, or vLLM traffic through it. It tracks usage per user and organization, caches responses, handles failovers between providers, and can block or redact requests containing personally identifiable information.

The interesting bit

Most API gateways handle authentication; this one counts tokens and watches your wallet. The failover cookbook shows you can chain providers—if OpenAI flakes, it falls back to Azure OpenAI—while still enforcing the same cost and rate limits. That turns a simple proxy into a production control plane for LLM sprawl.

Key highlights

• Per-key rate limits and cost caps (the README example sets a 25-cent spend limit)
• PII detection and masking, though it requires AWS credentials
• Provider failover and request retries with response caching
• Model-level and endpoint-level access control
• Datadog integration and privacy-controlled logging

Caveats

• PII detection requires AWS credentials, adding a cloud dependency to an otherwise self-hosted stack
• Admin server authentication is a single optional ADMIN_PASS environment variable, which feels thin for an “enterprise-grade” tool
• Needs PostgreSQL and Redis running alongside the gateway, so it is not a single-binary drop-in

Verdict

Worth a look if you are shipping LLM features to end users and need to meter usage or cap spend before someone loops a script overnight. Skip it if you just want a quick OpenAI proxy; the PostgreSQL and Redis requirements make this a small infrastructure project, not a sidecar.