LLMOps · Eval

A single Zig binary with an embedded Svelte dashboard that installs, supervises, and cross-wires local AI agents, workflow engines, and tracing tools so you don't have to juggle separate terminals.

VulnClaw exists so that a single natural-language sentence can trigger the entire reconnaissance-to-report pipeline without manually orchestrating a dozen separate tools.

A skill that spawns parallel reasoning processes under distorted cognitive frames, then scores and prunes them with a separate critic pass.

PhyAgentOS treats robot hardware like pluggable drivers so the same agentic session can run in simulation or on a real arm without rewriting the control stack.

Raven wraps agents in durable memory and self-refining skills so workflows survive past the chat session.

codex-keysmith exists because copying a Markdown file and editing one TOML key is simple, but existing files, active hooks, and interrupted writes are not.

iFixAi runs up to 32 inspections against any LLM or agent and returns a letter-grade scorecard in minutes, using a separate provider as judge so the model isn't grading its own homework.

Hallmark is a design skill that stops Claude, Cursor, and Codex from producing the same generic AI slop.

Agentlas OS compiles plain-language requests into portable, ownable agent packages that run locally across whatever LLM host you already use, while a Hub and owner-scoped Cloud handle sharing and retrieval.

It exists to stop your AI gateway from quietly burning through quotas, cash, and expired OAuth tokens without leaving a paper trail.

Token Monitor reads local logs from two dozen AI coding tools to surface live token burn, costs, and limits in one place, synced across all your machines.

Moss exists because calling out to a remote vector database adds 200–500 ms of latency—enough to kill a real-time conversation—so it runs embedding and search inside your process instead.

It packages a professor's decade of SIGMOD and NeurIPS experience into structured AI skills, bridging the 'last-mile' gap where generic guides and busy advisors leave grad students stranded.

This runtime layer gives Codex a structured, opt-in red-team workflow that stays out of your way until you explicitly enable it.

repowise indexes a codebase into five queryable intelligence layers—dependency graphs, git history, docs, architectural decisions, and deterministic health scores—so MCP-compatible agents can answer "why" instead of grepping for "what".

AIHelms wraps LiteLLM in a Vue management layer so finance can trace every token back to the department that spent it.

A hybrid CLI tool that uses deterministic pipelines to keep LLM agents from drifting off-target during code review.

Pairs a 424-page textbook with Jupyter notebooks to teach AI agent design patterns.

It chains a dozen LLM agents into a visual workflow so your on-call engineer only has to tap 'approve' instead of SSHing in at 3 AM.

Most AI scientist tools are monolithic prompt pipelines; FAROS treats research automation as a composable runtime problem rather than a single-agent stack.