Inference · Serving

OmniRoute keeps your coding agents online by automatically failing over across 177 AI providers—including free tiers—when quotas run dry.

AirLLM slices giant transformers into layer shards so they fit in consumer VRAM without quantization or distillation.

Pi bundles a unified LLM API, agent runtime, and interactive coding CLI into a monorepo that treats every dependency update as a potential attack.

It exists because clicking 'generate' isn't enough when you need to control every model, parameter, and preprocessing step.

Sub2API pools AI subscriptions behind a metered gateway so teams or resellers can distribute API quotas without building their own billing stack.

A Go proxy that handles OAuth login for Claude Code and Codex so you can call them through standard API clients.

It exists to run large language models on virtually any hardware—from Apple Silicon to RISC-V to your browser—with zero external dependencies and minimal setup.

Odysseus is a self-hosted AI workspace that replicates the ChatGPT experience on your own hardware while also handling email, calendars, documents, and deep research without shipping your data to third-party clouds.

Because juggling native APIs from a dozen LLM vendors, each with its own auth and billing, is a recipe for migraines.

vLLM is an open-source inference engine that pages attention key-value memory like an operating system to drive higher GPU throughput, then exposes it through an OpenAI-compatible API.

Because swapping from GPT-4o to Claude shouldn't require rewriting your request plumbing.

9Router is a local proxy that auto-switches your AI coding tools from paid to free providers and compresses token-heavy tool outputs so you stop hitting limits.

It exists so you can download, run, and chat with open-weight LLMs locally through one CLI and REST API, keeping inference on your own silicon.

SGLang exists to push low-latency, high-throughput inference for LLMs and multimodal models from a single GPU up to massive clusters.

A hosted proxy that offers free, rate-limited API access to GPT, DeepSeek, and others for Chinese users who'd rather not tunnel through a VPN.

OpenBB normalizes proprietary and public financial data so engineers can feed the same sources to Python scripts, REST APIs, Excel, and AI agents without rebuilding integrations.

It wraps local inference and fine-tuning for open models in a web UI, using custom kernels to squeeze more performance out of desktop GPUs than standard tooling.

VoxCPM2 proves TTS doesn't need discrete tokens: a 2B-parameter diffusion model generates continuous 48kHz speech for 30 languages and text-prompted voice cloning.

SillyTavern gives AI hobbyists a single, local interface to command text generators, image engines, and voice models across dozens of backends without losing control of their prompts or data.

A TTS project built to clone realistic voices from just one minute of training audio.