Berkeley's bet on RL for real-world LLM agents

What it does

SkyRL is a modular reinforcement-learning stack for LLMs, built by the Berkeley Sky Computing Lab. It covers training (skyrl-train), a local backend for the Tinker API (skyrl-tx), an agent layer for long-horizon tasks (skyrl-agent), and a gymnasium of tool-use environments (skyrl-gym) spanning math, coding, search, and SQL. The pieces are being folded into a single skyrl/ package, though the older paths still work.

The interesting bit

The project treats RL for LLMs as an infrastructure problem, not just an algorithmic one. It ships a unified engine that handles both training and inference, and it implements the Tinker API so you can run existing training scripts on local GPUs without rewriting them. The SQL pipeline is a notable proof point: a 7B model trained on 653 samples reportedly outperforms GPT-4o and o4-mini on Text2SQL, per the authors’ claims.

Key highlights

• Supports multi-turn, long-horizon agent training on real environments (e.g., SWE-Bench, terminal tasks)
• Implements the Tinker API for portable, hardware-local execution
• Includes pre-built Gymnasium environments for math, coding, search, and SQL
• Adopts patterns from veRL, OpenRLHF, Search-R1, and NVIDIA NeMo-RL
• Active downstream use: Biomni-R0, CodeScout, OpenThoughts-Agent, and others

Caveats

• The repo is mid-reorganization; the skyrl/ folder is the future, but migration is ongoing (see issue #1145)
• Exact reproduction of earlier SkyRL-v0 results requires checking out a specific commit

Verdict

Worth a look if you’re doing RL on LLMs with tool use, especially agents or SQL. Skip it if you just need a quick SFT wrapper or are allergic to moving parts.