A supervisor that learns from your agent's mistakes
Sentry sits beside your LLM agent, catches failures mid-run, guides the fix, and turns verified recoveries into reusable lessons.

What it does
Sentry is an external runtime layer that watches your agent’s action-observation cycles. It detects invalid actions, loop-stuck behavior, task drift, and unsupported assumptions, then either forces a retry in the required format (hard repair) or injects targeted guidance from previously learned lessons (soft repair). Crucially, it verifies the recovery actually worked before banking the lesson.
The interesting bit
Most runtime monitors hand out fixed advice forever. Sentry’s playbook grows online: every verified soft recovery becomes a reusable lesson that transfers to similar failures across tasks — without bloating the agent’s context. It’s the difference between a smoke alarm and a fire marshal who writes down what put the fire out.
Key highlights
- 37% average relative improvement over the strongest runtime-intervention baseline across WebShop, AppWorld, SWE-bench Lite, and Mind2Web Replay.
- 81.7% recovery rate across 939 detected failures.
- 1.54× base-agent token usage — versus 2.44× for AgentForesight.
- Actually lower average task latency than the base agent on SWE-bench Lite (31.7s saved) and AppWorld (45.6s saved).
- Provider-agnostic: OpenAI templates plus other providers and local endpoints.
Caveats
- The benchmark runner in this checkout only supports AppWorld dev tasks; the other paper benchmarks have no runners yet.
- Your application still owns action generation, validation, and execution — Sentry is guidance, not an executor.
Verdict
Worth a look if you run long-horizon agents that occasionally faceplant and you’d like them to get better at getting up. If your agent never fails, congratulations — you don’t need this.
Frequently asked
- What is nuglifeleoji/Sentry?
- Sentry sits beside your LLM agent, catches failures mid-run, guides the fix, and turns verified recoveries into reusable lessons.
- Is Sentry open source?
- Yes — nuglifeleoji/Sentry is open source, released under the MIT license.
- What language is Sentry written in?
- nuglifeleoji/Sentry is primarily written in Python.
- How popular is Sentry?
- nuglifeleoji/Sentry has 1.2k stars on GitHub.
- Where can I find Sentry?
- nuglifeleoji/Sentry is on GitHub at https://github.com/nuglifeleoji/Sentry.