← all repositories
nuglifeleoji/Sentry

A supervisor that learns from your agent's mistakes

Sentry sits beside your LLM agent, catches failures mid-run, guides the fix, and turns verified recoveries into reusable lessons.

★1.2k stars Python AgentsLLMOps · Eval
Sentry
Too new to rank. Velocity needs a few days of history.
not enough history yet
star history

What it does

Sentry is an external runtime layer that watches your agent’s action-observation cycles. It detects invalid actions, loop-stuck behavior, task drift, and unsupported assumptions, then either forces a retry in the required format (hard repair) or injects targeted guidance from previously learned lessons (soft repair). Crucially, it verifies the recovery actually worked before banking the lesson.

The interesting bit

Most runtime monitors hand out fixed advice forever. Sentry’s playbook grows online: every verified soft recovery becomes a reusable lesson that transfers to similar failures across tasks — without bloating the agent’s context. It’s the difference between a smoke alarm and a fire marshal who writes down what put the fire out.

Key highlights

  • 37% average relative improvement over the strongest runtime-intervention baseline across WebShop, AppWorld, SWE-bench Lite, and Mind2Web Replay.
  • 81.7% recovery rate across 939 detected failures.
  • 1.54× base-agent token usage — versus 2.44× for AgentForesight.
  • Actually lower average task latency than the base agent on SWE-bench Lite (31.7s saved) and AppWorld (45.6s saved).
  • Provider-agnostic: OpenAI templates plus other providers and local endpoints.

Caveats

  • The benchmark runner in this checkout only supports AppWorld dev tasks; the other paper benchmarks have no runners yet.
  • Your application still owns action generation, validation, and execution — Sentry is guidance, not an executor.

Verdict

Worth a look if you run long-horizon agents that occasionally faceplant and you’d like them to get better at getting up. If your agent never fails, congratulations — you don’t need this.

Frequently asked

What is nuglifeleoji/Sentry?
Sentry sits beside your LLM agent, catches failures mid-run, guides the fix, and turns verified recoveries into reusable lessons.
Is Sentry open source?
Yes — nuglifeleoji/Sentry is open source, released under the MIT license.
What language is Sentry written in?
nuglifeleoji/Sentry is primarily written in Python.
How popular is Sentry?
nuglifeleoji/Sentry has 1.2k stars on GitHub.
Where can I find Sentry?
nuglifeleoji/Sentry is on GitHub at https://github.com/nuglifeleoji/Sentry.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.