GRPO for agents without the GPU babysitting

What it does

ART trains multi-step LLM agents with GRPO through a client-server setup. Your application code drives the agent workflow, collects message trajectories, and assigns rewards when rollouts complete, while a GPU server manages vLLM inference and LoRA weight updates behind the scenes. It speaks an OpenAI-compatible API, so it drops into existing Python stacks without a rewrite.

The interesting bit

The split architecture is the practical twist: iterate on a laptop while the server handles ephemeral GPU training, or run both locally. That separation turns agent RL from an infrastructure project back into a modeling problem.

Key highlights

• Works with Qwen, Llama, GPT-OSS and others via vLLM with LoRA adapters
• Integrates with LangGraph and MCP servers, plus observability hooks for W&B and Langfuse
• Includes RULER for automatic reward generation and AutoRL for zero-data task training
• Ships with working notebooks for email search, 2048, Codenames, and Tic Tac Toe
• Supports SFT distillation and warmup workflows before RL kicks in

Verdict

Grab it if you are building multi-step Python agents and want to trade prompt-tuning for actual RL loops. Look elsewhere if you need a fully managed, no-code solution; ART still expects you to wire up trajectories and rewards.