LLMOps · Eval

Folkbench continuously probes API relay stations for availability, latency, and price, then lets you swap them into Claude Code or Codex with one click.

It stops long-running agents from drifting by forcing them to verify progress through independent roles before every new round.

Turns peer-reviewed research on AI-detectable narrative structure into a cross-platform agent skill that repairs fiction at the architectural level and tunes professional prose to its venue.

AnyJev turns any open LLM into a calibrated decision engine—typed choices with probabilities you can threshold—without touching a single model weight.

safelabs-eval fires a curated adversarial prompt corpus at your agent and scores the damage with pattern detectors — no grader model, no agent code changes.

A high-signal field guide that maps where Jev makes real typed decisions in production, with a strict warning that inclusion is not endorsement.

AiSOC is an open-source, self-hostable AI SOC that logs every LLM prompt and tool call so you can replay investigations instead of trusting a black box.

Log-based tools tell you what a session cost; cost-xray reads the raw API wire to show which part of the request — system prompt, MCP schemas, that one Read call — owns each token.

A cross-harness memory layer that captures sessions from Claude Code, Cursor, Codex and 20+ other tools, then turns the useful bits into reusable knowledge.

A prompt skill that removes generic AI phrasing from papers and grant proposals without flattening the scholarly voice or touching the evidence.

It packages a CRO specialist’s Twitter wisdom into a structured agent skill so your coding assistant stops building SaaS interfaces that look pretty but fail to convert.

One founder put his actual backlog and actual money on a public board to measure, honestly, whether AI agents can do paid commercial work.

This plugin exposes exactly what fills your agent's context window, when parts get compacted or pruned, and how the token budget is spent across every turn.

Agent Relay gives your coding agents channels, threads, DMs, and reactions so you can stop hand-rolling multi-agent plumbing.

ai-memory captures session context from Claude, Codex, and others into a git-backed markdown wiki so you can quit one agent and resume in another without starting from scratch.

A local-first desktop workbench that structures the entire novel pipeline—from premise to final draft—so your AI doesn’t lose track of the plot by chapter three.

A Claude Code skill that borrows aerospace maintenance-English discipline to stop AI agents from misreading each other's tool descriptions and error messages.

It treats automated short-video production as a resilient microservice factory, not a one-off Python script.

ripwire builds ranked, deterministic repo maps so AI agents can navigate code without the token bloat of grep-and-read tours.

This runtime breaks complex AI tasks into isolated, versioned nodes so you can resume from failure instead of rerunning the entire chain.