Route 100+ LLMs through one OpenAI-shaped hole

What it does Helicone sits between your application and whatever LLM you’re calling. Change the baseURL and it proxies your traffic, logging traces, costs, and latency while letting you swap models or providers on the fly. It bundles observability, prompt versioning, a testing playground, and analytics into one dashboard—offered as a managed service or fully self-hosted via Docker.

The interesting bit The architecture is pleasantly paranoid: it runs on Cloudflare Workers at the edge for proxy logging, shoves analytics into ClickHouse, and stores logs in MinIO, all wired together with Supabase. That stack means you can self-host the entire thing locally if you don’t want your prompts leaving your network.

Key highlights

• AI Gateway exposes 100+ models through a single OpenAI-compatible API key, with automatic fallbacks and intelligent routing.
• One-line integrations cover OpenAI, Anthropic, LangChain, Gemini, Vercel AI SDK, and a long tail of inference providers.
• Prompt management versions your production prompts and lets you deploy changes through the gateway without touching application code.
• SOC 2 and GDPR compliant, with a free tier of 10,000 requests per month.
• Self-hosting is first-class: Docker Compose is documented, and a production Helm chart is available for enterprise workloads.

Caveats

• The Helm chart requires contacting the team directly; it isn’t in the public repo.
• Fine-tuning is currently limited to partner integrations (OpenPipe and Autonomi), with more promised but not yet available.

Verdict Worth a look if you’re running multi-provider LLM workloads and need cost visibility or want to self-host your observability stack. Skip it if you just need a simple OpenAI logger—this is overkill for a single provider.