Reinforcement learning that refuses to pick a favorite framework

What it does

skrl is a modular Python library for reinforcement learning that implements algorithms across three numerical backends: PyTorch, JAX, and NVIDIA Warp. It wraps popular environment interfaces—Gymnasium, PettingZoo, ManiSkill, NVIDIA Isaac Lab, and MuJoCo Playground—into a common training structure. The library emphasizes readable, transparent algorithm code over black-box abstractions.

The interesting bit

The standout feature is “scopes”: you can partition environments into subsets that train simultaneously in the same run, optionally sharing resources or staying isolated. That means you could mix Isaac Lab and MuJoCo Playground rollouts under one training loop without maintaining separate scripts. It is the kind of flexibility that sounds bureaucratic until you are juggling multiple simulators.

Key highlights

• Implements RL algorithms in PyTorch, JAX, and NVIDIA Warp from a single API
• Supports Gymnasium, PettingZoo, ManiSkill, NVIDIA Isaac Lab, and MuJoCo Playground
• Enables simultaneous training across environment “scopes”—subsets that may or may not share resources
• Published in JMLR with separate CI test suites for each backend
• Hosts pretrained models on Hugging Face

Verdict

If you want to prototype RL agents across PyTorch, JAX, and Warp without rewriting training loops for every backend, skrl offers a unified modular API. If you need a comprehensive list of implemented algorithms or performance baselines, you will have to leave the README and consult the external documentation.