Inference · Serving

OmniRoute keeps your coding agents online by automatically failing over across 177 AI providers—including free tiers—when quotas run dry.

Pi bundles a unified LLM API, agent runtime, and interactive coding CLI into a monorepo that treats every dependency update as a potential attack.

It catalogs legitimate services offering free API access to large language models, complete with rate limits, model lists, and data-privacy caveats.

A minimal C/C++ port of OpenAI’s Whisper built to transcribe speech locally on phones, browsers, and underclocked POWER9 boxes.

It wraps local inference and fine-tuning for open models in a web UI, using custom kernels to squeeze more performance out of desktop GPUs than standard tooling.

9Router is a local proxy that auto-switches your AI coding tools from paid to free providers and compresses token-heavy tool outputs so you stop hitting limits.

A hosted proxy that offers free, rate-limited API access to GPT, DeepSeek, and others for Chinese users who'd rather not tunnel through a VPN.

Mozilla wraps llama.cpp and a full model into a single cross-platform executable using an obscure libc trick.

It centralizes model definitions so the same architecture works across PyTorch, JAX, vLLM, and llama.cpp without rewrites.

LibreChat bundles every major LLM provider into a single self-hosted chat platform so teams don't have to choose—or leak data.

Because swapping from GPT-4o to Claude shouldn't require rewriting your request plumbing.

Microsoft built an inference engine that lets a single CPU run a 100B-parameter model at human reading speed by using 1.58-bit weights.

Sub2API pools AI subscriptions behind a metered gateway so teams or resellers can distribute API quotas without building their own billing stack.

It wraps Stable Diffusion—and the whole messy ecosystem of models, upscalers, and LoRAs—into one browser-based workbench.

LocalAI wraps 36+ inference engines behind one OpenAI-compatible API and pulls them on demand, so you can run LLMs, vision, voice, and video on anything from a CPU to a Jetson.

DeepSeek-V3 exists to prove that a 671-billion-parameter model can train end-to-end without a single rollback, activate only 37B parameters per token, and still match leading closed-source systems.

An OCR model that asks how few vision tokens an LLM needs before it can no longer read the page.

Because training a transformer shouldn't require 245MB of PyTorch just to multiply matrices.

It exists to let developers run customized vision, text, and audio machine learning across mobile, web, and edge hardware without cloud round-trips.

YOLOv5 made real-time object detection as easy as `torch.hub.load`, then exported to everything from iOS to edge chips.