← all repositories
mozilla-ai/otari

Mozilla's self-hosted answer to LiteLLM

An OpenAI-compatible gateway you run yourself — one endpoint for 40+ providers, virtual keys instead of real ones, and budgets checked before the money moves.

otari
Velocity · 7d
+4.1
★ / day
Trend
↘cooling
star history

What it does Otari sits between your applications and model providers. It authenticates each request, resolves the real provider credentials server-side, checks budgets before dispatch, and records usage after settlement — so your apps talk to one OpenAI- and Anthropic-compatible endpoint and never touch a real provider key. Provider calls route through mozilla-ai’s any-llm library. Run it standalone with your own database, or wire it to otari.ai as a hosted control plane.

The interesting bit The boring part is the value: cost control lives in the gateway itself — budgets enforced before spend, usage recorded after settlement — rather than in a cron job you’ll forget to write. Routing policies cover failover, weighting, and “learned selection,” which the README names but never explains, so file that under promising-but-undocumented. There’s also a guardrails hook that rides along with coding agents like Claude Code without needing the gateway itself.

Key highlights

  • One OpenAI- and Anthropic-compatible endpoint (chat/completions, messages, responses) fronting 40+ providers
  • Revocable virtual keys scoped to user, workspace, or model — real credentials stay behind the gateway
  • Budgets enforced before dispatch; usage recorded after settlement
  • Optional services: code-execution sandbox, web search, MCP, inference guardrails, file understanding
  • A dashboard for providers, routing, keys, members, budgets, and usage, with Swagger UI served live

Caveats

  • “Learned selection” routing is mentioned but never explained — unclear what’s learned or how
  • Hybrid mode reports usage to otari.ai; fine when you opt in, but know where your usage data goes
  • Python 3.13+ if you run from source — a fairly fresh requirement

Verdict Teams juggling multiple providers who want key hygiene and spend ceilings without routing traffic through someone else’s SaaS should look closely. One provider, one app? This is an extra moving part you don’t need.

Frequently asked

What is mozilla-ai/otari?
An OpenAI-compatible gateway you run yourself — one endpoint for 40+ providers, virtual keys instead of real ones, and budgets checked before the money moves.
Is otari open source?
Yes — mozilla-ai/otari is open source, released under the Apache-2.0 license.
What language is otari written in?
mozilla-ai/otari is primarily written in Python.
How popular is otari?
mozilla-ai/otari has 959 stars on GitHub and is currently cooling off.
Where can I find otari?
mozilla-ai/otari is on GitHub at https://github.com/mozilla-ai/otari.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.