Your agent loop is an inference workload, not just a prompt

What it does

Inferoa is a TypeScript CLI and TUI that runs recursive, long-horizon agent loops—what the authors call “Loop Engineering.” It gives the model an objective, tools, verification steps, and memory, then lets it self-correct across turns until the work is proven. Unlike a simple chat wrapper, it treats the loop itself as an inference workload that needs runtime discipline.

The interesting bit

The project is essentially an opinionated harness built atop the vLLM ecosystem. It obsesses over “tokenmaxxing”: preserving reusable prompt prefixes, bounding mutable context, and routing requests to the cheapest adequate model path—whether that is a self-hosted vLLM endpoint or a frontier API—based on session pressure, safety, and capability needs.

Key highlights

• /loop spawns durable recursive loops that carry objectives across tasks, attempts, verification, and recovery rather than stopping at the first answer.
• Prefix-cache discipline uses prompt epochs and deterministic tool schemas to keep reusable prefixes intact as turns accumulate.
• Context optimization pulls in graph-shaped repo context and bounded history to prevent stale evidence from consuming the window.
• Intelligent model selection can route to self-hosted vLLM, vLLM-Omni, or external APIs depending on cost, privacy, and multimodal needs.
• A /tokenmaxxing command surfaces live token pressure, prefix-cache reuse stats, and model-selection overhead inside a session.

Caveats

• The README is heavy on framework jargon (“Loop Engineering,” “tokenmaxxing”) and light on concrete benchmarks or comparisons to existing agent frameworks.
• It is tightly coupled to the vLLM ecosystem; if you are not running vLLM or using its semantic router, several of its routing and serving benefits may not apply.

Verdict

Worth a look if you are running self-hosted vLLM and need agents that work for hours without silently turning your context window into a landfill. Skip it if you are looking for a model-agnostic, quick-drop-in agent framework with no interest in serving infrastructure.