A flight recorder for multi-agent LLM apps

What it does

AgentOps provides a Python SDK and optional self-hosted dashboard for monitoring AI agents in the wild. It instruments your code to record step-by-step execution graphs, tracks spending across LLM providers, and captures inputs, outputs, and exceptions. The goal is to move beyond printf debugging when a multi-agent workflow veers off course.

The interesting bit

The project meets developers where they are: instead of forcing a single framework, it offers native integrations with CrewAI, AG2, LangChain, Camel AI, and the OpenAI Agents SDK, plus decorator-based tracing (@session, @agent, @operation) for custom code. You can use the hosted dashboard or run the entire stack—API and frontend—on your own infrastructure.

Key highlights

• Supports both hosted and fully self-hosted deployments under the MIT license
• Decorator-based instrumentation (@session, @agent, @workflow) with async, generator, and nested span support
• Native integrations with CrewAI, AG2 (AutoGen), LangChain, Camel AI, and OpenAI Agents SDK (Python and TypeScript)
• Session replay with step-by-step execution graphs and chat viewer for debugging agent decisions
• LLM cost tracking across foundation model providers

Verdict

Worth a look if you are running non-trivial agent workflows and need to trace failures or audit spend. Probably overkill if you are just calling openai.chat.completions.create in a script and calling it a day.