The MLOps loop for agents that learn on the job

What it does Reef sits between your inference engine and training framework to run a continual learning cycle: it serves agent requests, matches feedback to specific interactions, produces model-weight or harness updates via pluggable recipes, and commits only evaluated winners to a version history. The system stays live through updates, so new weights or prompt skills reach the runtime without a restart.

The interesting bit Most agent infra stops at deployment; Reef treats live traffic as a training signal. It supports two distinct surfaces—model weights via Slime and SGLang, and agent harnesses like prompts and skills via pure API calls—under the same versioning and evaluation gate. The README is admirably concrete about which recipe fits which feedback pattern, from scalar scores to next-state signals.

Key highlights

• Closed four-step loop: serve traffic, observe feedback, grow candidates, and commit evaluated updates to versioned artifacts.
• Dual learning surfaces: train actual model weights when you have GPUs, or evolve harnesses (prompts, rules, skills) using only an upstream model API.
• Recipe-based extensibility: swap in SAO, OpenClaw-RL, TTT-Discover, SkillClaw, or GEPA depending on whether your feedback is explicit scores, next-state signals, or transcripts.
• OpenAI- and Anthropic-compatible inference endpoint that returns interaction receipts, letting you attach scores or structured feedback later.
• Rejected update candidates stay out of the release; the serving runtime keeps running the current version while evaluation happens elsewhere.

Caveats

• Artifact and checkpoint storage depends on git-lfs, so you’ll need that system package installed.
• Training examples and recipe implementations live in the source repository and do not ship in the PyPI wheel; plan on working from a checkout.
• Weight-training deployments need a supported GPU stack and a trainable model, so this isn’t a drop-in SaaS wrapper.

Verdict Worth a look if you run persistent agents and want them to get better from real interactions rather than batch retraining. Skip it if you just need a one-shot inference proxy with no learning loop.