Turning 100 LLM dialects into one OpenAI accent

What it does LiteLLM is an open-source AI gateway that speaks OpenAI’s API format to practically every major LLM provider—Anthropic, Gemini, Bedrock, Azure, and dozens more. Use it as a Python SDK or deploy it as a self-hosted proxy so your entire team hits one endpoint instead of juggling provider-specific SDKs, auth patterns, and error shapes.

The interesting bit What started as a simple API shim has grown into full-blown infrastructure middleware: it now brokers A2A agents and MCP tool servers, effectively treating OpenAI’s request format as a universal protocol for chat, embeddings, images, and even agent-to-agent traffic. It’s either ambitious scope creep or a bet that the future of AI integration looks a lot like a standardized HTTP proxy.

Key highlights

• Supports 100+ providers across chat, embeddings, images, audio, batching, reranking, and agent endpoints
• Production gateway features: virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard
• Claims 8 ms P95 latency at 1,000 RPS (see project benchmarks)
• Adopted by Stripe, Netflix, Google ADK, and OpenAI Agents SDK
• Bridges MCP servers and A2A agents into the same OpenAI-compatible request flow

Caveats

• The MCP client is explicitly named experimental_mcp_client, so that surface is still settling
• Provider coverage varies by endpoint; the full matrix is large but not every provider supports every modality

Verdict Teams running multi-provider LLM workloads in production should look here; hobbyists calling a single OpenAI endpoint probably don’t need the extra hop.