An AI gateway that treats context windows like a budget, not a landfill

What it does

Krebs is a self-hosted AI gateway built around a stateful, general-purpose agent. It offers multi-turn dialogue, Lua tool execution, subagent orchestration, and persistent sessions, all reachable through a browser UI or HTTP and WebSocket APIs. You can swap between DeepSeek and Claude backends by changing a single environment variable.

The interesting bit

Instead of treating context windows as an ever-growing scrapbook, Krebs runs a layered compression pipeline that prunes, summarizes, and retrieves memories at specific token thresholds. It also patrols for conversational drift: a goal-constraint engine and a post-response self-verification loop inject correction messages when the agent starts answering the wrong question.

Key highlights

• Five-stage context compression, from BM25 session retrieval at the perception phase down to auto-compaction when usage hits 83.5%
• Two-phase memory: at 50% token usage the LLM appends a summary to MEMORY.md, which is reinjected into the system prompt on the next session start
• A WASM sandbox using wasmtime and coreutils.wasm that traps write commands to the working directory while letting reads passthrough to bash
• Drift detection via BM25 hybrid scoring (keyword 0.6, semantic 0.4) that triggers [GOAL CONSTRAINT] corrections, plus a post-response verifier with up to five retries
• Subagent fleet management with task queues, scheduled cron-style jobs, and live fleet monitoring

Caveats

• The WASM sandbox explicitly forbids pipes, redirects, and command chaining, so shell workflows are constrained to simple one-shot operations
• Session-history retrieval automatically disables itself when context is already more than 80% full, leaving the agent without past-session grounding precisely when the window is most crowded
• The README’s architecture diagram is truncated, so some wiring details remain opaque

Verdict

Worth a look if you run long-lived autonomous agents and spend too much time manually nudging them back on topic. If you just need a quick chat wrapper, the complexity is overkill.