Inference · Serving

A demo repo for running extreme-quantized language models locally without needing a research cluster.

AOS Community Edition is a Rust-based agent operating system that lets agents inspect the runtime, spot missing capabilities, and forge their own least-privilege extensions.

An unofficial proxy that borrows your ChatGPT/Codex OAuth tokens to serve a local OpenAI-compatible API, bypassing API credit billing.

It exists to stop your AI gateway from quietly burning through quotas, cash, and expired OAuth tokens without leaving a paper trail.

It keeps the entire agent loop—prompts, tool calls, browser state, and memory—on your laptop so you don't have to rent a control plane in the cloud.

A Go proxy that tricks Claude Code into using $5/month open models through OpenCode instead of Anthropic's API.

It breaks the vendor lock on Codex and Claude Code by translating their API calls to any LLM backend you choose.

A self-hosted framework for building real-time voice-first AI agents that persist memory, delegate long tasks to background sub-agents, and optionally show up as lip-synced digital humans.

Espressif's C framework turns cheap microcontrollers into edge AI agents you program through IM chat.

TokenHub exists because dropping a shared OpenAI key into a Slack channel does not scale past one invoice and zero accountability.

One desktop UI that wires together ComfyUI, OpenAI, Gemini, ModelScope, and a dozen other generative APIs—plus some very opinionated legal terms.

It split off from `verl` to give diffusion, video, and omni-modality models an RL post-training framework that doesn't treat them like chatbots.

OpenLake wants storage to bypass the host entirely and land straight in GPU memory.

turbovec exists so you can index embeddings immediately—no training, no tuning, no rebuilds—and search them faster than FAISS in a fraction of the RAM.

It exists because cloud image generators deserve a local memory layer, a branching canvas, and a UI outside the chat thread.

Dream Server exists because most people would rather pay OpenAI than spend a weekend hand-wiring Docker configs for local LLMs, RAG, and image generation.

It unifies Stable Diffusion, GGUF chat, Whisper, and Kokoro TTS into a single offline desktop GUI so you can skip cloud APIs, subscriptions, and censorship filters.

A Bittensor subnet that uses zero-knowledge proofs to verify miners actually ran the AI models they claim to.

AngelSlim integrates quantization, speculative decoding, and distillation so you can shrink and serve massive models from a single toolkit.

KTransformers makes CPU-GPU heterogeneous inference and fine-tuning for massive MoE models almost practical on consumer hardware.