Denys88/rl_games
A battle-tested RL library that keeps the GPU busy
rl_games is a PyTorch reinforcement learning framework built for high-throughput, GPU-accelerated training across simulators like Isaac Gym, Brax, and MuJoCo.
Velocity · 7d
+0.5
★ / day
Trend
→steady
star history
What it does
rl_games implements PPO and related algorithms with a focus on keeping the entire training pipeline on the GPU. It supports asymmetric actor-critic setups, masked actions, multi-agent training with centralized or decentralized critics, and self-play. The library wraps a wide range of environments—Isaac Gym, Brax, EnvPool, Atari, StarCraft II, DeepMind Control Suite—behind a unified config-driven runner.
The interesting bit
The project has real research mileage: it backs published work on dexterous manipulation (DeXtreme, DexPBT) and quadruped locomotion, and the authors actively maintain compatibility with the shifting ecosystem—most recently migrating from gym to gymnasium and restoring EnvPool support for NumPy 2.x and MuJoCo 3.x. That is the unglamorous work that keeps a library usable.
Key highlights
- End-to-end GPU training with Isaac Gym and Brax; observations and actions never leave the device
- Multi-GPU via
torchrun, experiment tracking via Weights & Biases - ONNX export for trained models, including LSTM variants
- Optional extras for
atari,mujoco,envpool,brax,pufferlib - Colab notebooks for MuJoCo, Brax, and ONNX export quickstarts
Caveats
- TensorFlow 1.x implementations (Rainbow DQN, A2C, PPO) were removed; only PyTorch remains
- Version 2.0.0 is unreleased, so the
gymnasiummigration is not yet in a stable tag
Verdict
Worth a look if you are doing GPU-heavy RL research in robotics or need a single framework that spans Isaac Gym to Atari. Probably overkill if you just want to train a CartPole baseline.