A drop-in RL trainer for agents that respects your stack

What it does

Agent Lightning is a training harness that wraps around existing AI agents to optimize them with reinforcement learning, automatic prompt optimization, and supervised fine-tuning. You keep your current stack—LangChain, CrewAI, AutoGen, raw OpenAI calls, or Microsoft’s own—and add lightweight instrumentation via agl.emit_xxx() or a tracer that streams prompts, tool calls, and rewards into a central LightningStore. A Trainer feeds those traces to an algorithm of your choice, which posts back updated weights or refined prompts, closing the loop without a rewrite.

The interesting bit

The “almost zero code change” promise works by treating your live agent as the environment itself, rather than forcing you to reimplement it inside a new RL sandbox. You can also selectively optimize individual agents inside a multi-agent swarm instead of treating the whole orchestration as one opaque policy.

Key highlights

• Framework-agnostic by design: works with LangChain, OpenAI Agent SDK, AutoGen, CrewAI, or plain Python OpenAI calls.
• Supports selective optimization inside multi-agent systems, so you can tune one agent without destabilizing the others.
• Algorithm buffet includes RL, automatic prompt optimization, and supervised fine-tuning; you can also plug in your own.
• Verified at scale by downstream projects (e.g., Tencent’s Youtu-Agent) running steady convergence on 128 GPUs for math, code, and search tasks.

Caveats

• The README promises “ZERO CODE CHANGE (almost)"—the parenthetical is doing real work, and the exact surface area of those changes is unspecified beyond dropping in agl.emit_xxx() or enabling a tracer.
• The architecture diagram and store abstraction look clean, but the README does not quantify overhead latency or memory footprint for the LightningStore and Trainer loop.

Verdict

Worth a look if you already have agents in production and want to experiment with RL or prompt optimization without a ground-up rebuild. Skip it if you are starting from scratch and want an opinionated, all-in-one agent framework rather than a training harness.