Living Off the Land: The Router That Turns Free Tiers Into One API

Every serious AI lab now hands out a free tier like a sample spoon at an ice cream counter: a few million tokens a month, a few thousand requests a day, just enough to taste. On its own, each tier is a toy. But FreeLLMAPI, a self-hosted proxy from Tashfeen Ahmed, starts from a simple observation: stack the free tiers from Google, Groq, Cerebras, SambaNova, Mistral, OpenRouter, GitHub Models, Cohere, Cloudflare, HuggingFace, Z.ai, Ollama, Kilo, Pollinations, LLM7, and NVIDIA, and you get roughly 1.7 billion tokens per month of working inference capacity across more than a hundred models. The project’s entire reason for existing is the gap between that aggregate number and the miserable experience of actually using it — sixteen SDKs, sixteen rate-limit regimes, sixteen places a request can die mid-task.

The pitch is one OpenAI-compatible endpoint. Point any OpenAI client library at your local server, and a router picks the best available model for each request, falls over to the next provider when one is rate-limited, and tracks per-key usage so you stay under every free-tier cap simultaneously. Your apps authenticate with a single unified bearer token; the upstream provider keys never leave the proxy, and they’re encrypted with AES-256-GCM before touching SQLite.

The boring part is the product

Strip away the provider logos and FreeLLMAPI is a rate-limit ledger with an HTTP server attached. That is not a dismissal — it’s the whole insight. The hard problem in aggregating free tiers isn’t translation between API dialects (most providers now speak OpenAI’s wire format natively, and the rest get adapters). The hard problem is bookkeeping: knowing, at the moment a request arrives, which of your keys is under its requests-per-minute, requests-per-day, tokens-per-minute, and tokens-per-day caps — all four, per platform, per model, per key — and which ones are sitting in a cooldown after a 429.

The router maintains exactly those counters, backed by SQLite, and picks the highest-priority model that is both healthy and under all its limits. When a provider returns a 429, a 5xx, or times out, the key goes on a short cooldown and the request retries down the fallback chain — up to twenty attempts. Periodic health probes mark keys as healthy, rate-limited, invalid, or errored, so the router skips dead ones without waiting for a request to fail first.

One design detail deserves more attention than it usually gets: sticky sessions. Multi-turn conversations keep talking to the same model for thirty minutes, specifically to avoid the quality cliff that comes from switching models mid-conversation. Anyone who has watched a coding assistant abruptly forget its own conventions after a silent model swap understands why this matters. The fallback chain is a safety net, not a round-robin.

The Responses API — the wire format current Codex CLI versions require — is implemented as a translating shim over the same router, with streaming events and tool calls. Tool calling passes through natively for OpenAI-compatible providers; Gemini requests get translated into Google’s functionDeclarations shape and back. Every response carries an X-Routed-Via header so you can see which provider actually served the call, plus a fallback-attempt count when a request hopped providers. That last header is quietly the most honest feature in the project: it refuses to pretend the abstraction is seamless.

The candor is unusual

Two sections of the README distinguish FreeLLMAPI from most aggregator projects. The first is a Terms of Service review — a per-provider table, re-reviewed in May 2026, grading each upstream’s terms from “likely OK” to “avoid.” Cohere gets an avoid because its terms still forbid personal use. NVIDIA NIM is disabled by default because its trial terms restrict use to evaluation. Google’s March 2026 ToS narrowing to “professional or business purposes” gets flagged as a caution. Z.ai’s anti-traffic-redirect clause is noted as plausibly readable against a proxy. This is the unglamorous legal homework that most projects skip entirely, and it matters here more than usual, because the entire premise is arbitrage against sixteen different contracts simultaneously.

The second is the limitations section, which reads like it was written by someone who has actually run the thing. “No frontier models” — the free-tier catalog tops out around Llama 3.3 70B, GLM-4.5, and Gemini 2.5 Pro. “No SLA, by definition.” “Local-first” — no multi-tenant auth, don’t expose it to the internet. And then the genuinely interesting one: intelligence degrades as the day progresses. Your best models have the lowest daily caps. Once they’re exhausted, the router slides down your priority chain to smaller models, and the effective intelligence of your endpoint drops through the late hours before resetting at UTC midnight. That’s an emergent property of free-tier economics that no single provider’s dashboard would ever show you, and it’s the kind of observation that only comes from living inside the constraint.

A good moment for free tiers

The timing isn’t accidental. Free LLM access in 2026 is, by historical standards, genuinely good — a Dataiku survey attributes this to competitive pressure from Meta’s open-weight Llama 4 release and DeepSeek’s V3 pushing inference costs down, and notes that all five of the major free providers it examined support OpenAI-compatible SDKs, making switching a one-line change rather than a rewrite. NVIDIA’s NIM catalogue alone offers 120+ open-weight models — DeepSeek, Kimi, GLM, Qwen, Nemotron — through a single OpenAI-compatible endpoint, explicitly as a shop window for its enterprise product. There’s even a curated awesome-free-llm-apis list tracking permanent free tiers, whose commit history reads like a field report from the front: Groq’s rate limits corrected, Mistral models retired, OpenRouter swapping seven free models out and nine in, Cerebras quietly requiring a payment method.

That last repository also carries a cautionary tale for this whole corner of the ecosystem: its most recent commit removed a malicious security-audit workflow described as a credential stealer. When the currency of a scene is API keys, the scene attracts key thieves. FreeLLMAPI’s encrypted-at-rest key storage and unified-key auth model — where apps never see upstream credentials — is a reasonable architectural answer to exactly that threat.

Where it sits in the gateway landscape

FreeLLMAPI is entering a crowded field, and it’s honest about occupying the cheap seats. The unified LLM API market has consolidated around the OpenAI-compatible interface as the de facto standard, which means competition happens on surrounding features: caching, cost visibility, observability, fallback behavior. Production-grade gateways like LLM Gateway — 40+ providers, 250+ models, SAML SSO, PII guardrails, spend caps, customers like Samsung and Harvard — and the gateways surveyed by Agenta (LiteLLM, Helicone, Kong) all solve the same operational problems: rate limits, provider outages, model churn, key management. FreeLLMAPI deliberately doesn’t compete on any of that. It’s single-user by design, no multi-tenant billing, no enterprise anything. Compared to OpenRouter’s hosted free routes or LiteLLM’s production orientation, it targets the individual developer who wants the routing without the vendor.

It’s also, fundamentally, glue code — and good glue. The value isn’t in any novel algorithm; it’s in the maintained catalog of free-tier models, the per-provider ToS vigilance, and the rate-limit ledger that makes sixteen caps behave like one budget. Glue that saves you from maintaining sixteen integrations yourself is worth exactly as much as the maintenance burden it removes, and no more.

One wrinkle worth flagging: the sources disagree on the project’s own scale. The README describes sixteen providers and ~1.7 billion tokens per month; a Betterstack guide describes 28 providers, 339 free model endpoints, roughly 4 billion tokens, and features the README doesn’t mention — routing strategies (Balanced, Smartest, Fastest, Most Reliable) with weighted scoring, and setup generators for Claude Code, Codex, and DeepSeek Harness. Either the project is moving faster than its README, or the two sources describe different points in its evolution. The Betterstack description is the more ambitious one; the README is the more conservative one. Which is current is unclear, but the direction of travel — more providers, smarter routing — is consistent.

The treadmill problem

The project’s long-term vulnerability isn’t technical. It’s that free tiers are policy, and policy changes without notice. The awesome-list’s commit log — models retired, limits corrected, providers moving to paid-only — is a preview of the maintenance treadmill FreeLLMAPI is on forever. Moonshot and MiniMax direct integrations have already been dropped from the catalog; HuggingFace was removed over tool-call format issues. When a provider tightens its terms, users see 429s and auth errors until someone updates the catalog and re-seeds the models. The ToS review table, with its May 2026 re-review date, is effectively a promise to keep doing this homework indefinitely — and the moment that homework lapses, the project’s core value proposition quietly rots.

There’s also an unresolved tension in the premise itself. The disclaimer is explicit: this is for personal experimentation, not production, and if you build something real on it, swap in a paid API before you ship. But the entire engineering effort — fallover chains, health checks, analytics dashboards, sticky sessions — is reliability engineering for a substrate the project itself says isn’t reliable. That’s not incoherent; it’s a reasonable bet that “good enough for prototyping, free forever” is a real niche, and that the difference between a toy and a tool is whether it dies at 2 p.m. when Gemini’s daily cap runs out.

For the developer with a side project, a hackathon entry, or a coding assistant habit that doesn’t justify a monthly bill, that bet looks sound. FreeLLMAPI won’t give you frontier reasoning, won’t give you an SLA, and won’t stay smart after sundown. What it will do is make sixteen free tiers behave like one API that mostly works — and be unusually honest about the ways it doesn’t.