The offline RL benchmark is breaking up — on purpose

What it does

D4RL is a reference benchmark for offline reinforcement learning: it bundles control, navigation, manipulation, and driving tasks with pre-recorded datasets so algorithms can be trained without live environment interaction. Everything exposes the OpenAI Gym API, and each task includes a fixed dataset of observations, actions, rewards, and termination flags that downloads automatically. The project also supplies normalized scoring utilities so researchers can compare episode returns on a common scale.

The interesting bit

The most striking thing about D4RL is that it is actively being decommissioned. The Farama Foundation is pulling environments out into Gymnasium, MiniGrid, and Gymnasium-Robotics, while datasets are migrating to a new standard repository called Minari; the README leads with a warning to upgrade because this repo is now in maintenance mode with long-standing bugs.

Key highlights

• Environments span locomotion, maze navigation, dexterous manipulation, and autonomous driving (via MuJoCo, CARLA, and other backends)
• Every task ships with a fixed offline dataset accessible through env.get_dataset(), plus a qlearning_dataset() helper that adds next_observations
• Includes env.get_normalized_score() to compare undiscounted episode returns against reference min/max values
• Datasets are licensed CC BY and code is Apache 2.0

Caveats

• Active development, bug fixes, and new Python support have moved to Gymnasium, MiniGrid, Gymnasium-Robotics, and Minari; this repository is frozen in maintenance mode
• PyBullet and Flow environments are explicitly abandoned and will not be maintained or ported
• Control tasks may still require a MuJoCo license and mujoco_py, and the last substantive update was in 2020

Verdict

Reach for D4RL only if you need to reproduce historical paper results that report its normalized scores; start new offline RL work in Minari and Gymnasium-Robotics instead.