A retired toolbox that taught robots to learn what matters

What it does

S-RL Toolbox was a research framework that married ten RL algorithms (PPO, SAC, DDPG, etc.) with state-representation learning methods for robotics. It shipped with PyBullet simulation environments—Kuka arm variants, mobile robots, a racing car—plus real-robot support for Baxter and ROS-based setups. The goal: test whether learning a compact “understanding” of the world from camera pixels helps robots reach targets more efficiently than end-to-end RL.

The interesting bit

The authors decoupled feature extraction from policy learning, letting you swap in different SRL methods (autoencoders, forward models, etc.) and compare them head-to-head. The environments expose “ground truth” robot position as an alternative to pixels, so you can measure exactly how much your fancy representation is costing you in sample efficiency.

Key highlights

• 10 RL algorithms integrated via Stable Baselines, with hyperparameter search (Hyperband, Hyperopt)
• Custom Gym environments running at ~250 FPS in PyBullet on an 8-core machine
• One-line switch between raw pixels and ground-truth state with --srl-model
• Docker and Anaconda setup; claimed 1M steps/hour on 8-core CPU + Titan X GPU
• Published research with arXiv papers and video demos

Caveats

• Explicitly unmaintained; the authors redirect to Stable-Baselines3 and RL Baselines3 Zoo
• Python 3 only; depends on the older stable-baselines fork, not the modern SB3
• Real-robot integration (Baxter, Robobo) requires ROS, adding setup friction

Verdict

Worth a look if you’re reproducing 2018 SRL-for-robotics papers or studying how representation learning affects sample efficiency. Skip it if you want production RL code—use Stable-Baselines3 instead.