Robot RL that stops begging for GPU time

What it does

UniLab is a reinforcement-learning training stack for robotics that deliberately splits work across heterogeneous hardware. Physics simulation runs on CPU (via MuJoCo or MotrixSim), then streams experience through a shared-memory replay buffer to GPU policy training (PPO, SAC, TD3, APPO, and others). The project explicitly targets setups that aren’t NVIDIA-only: Linux ROCm, Intel XPU, and Apple Silicon are all tracked as first-class setup paths.

The interesting bit

The architecture inverts the typical GPU-everything assumption in RL. By keeping physics on CPU and using unified shared memory as the coupling layer, UniLab lets you train on a MacBook or an AMD box without pretending the simulator can run on your graphics card. The task=<task>/<backend> Hydra config pattern makes switching between MuJoCo and MotrixSim a one-token change.

Key highlights

• Supports MuJoCoUni and MotrixSim backends through adapter-based integration
• Policy algorithms include PPO, MLX PPO, APPO, SAC, TD3, FlashSAC, with HORA and HIM-PPO as script-level workflows
• Cross-platform setup tracked for Linux CUDA, ROCm, XPU, and macOS / Apple Silicon
• Pre-trained demo checkpoints (dance, wallflip, in-hand grasp, loco-manipulation) auto-download from Hugging Face
• Hydra-based “task owner” configs bundle task, reward, backend, and algorithm selection in one YAML file

Caveats

• The README notes that Conda and pip users “should still follow the uv workflow for now,” so dependency management is effectively uv-mandated
• Grasp caches and demo assets pull from Hugging Face on first run; mainland China users need to redirect to a mirror (hf-mirror.com)
• Some algorithm paths (HORA, HIM-PPO) are documented as script-level workflows rather than first-class CLI commands

Verdict

Worth a look if you’re doing robot RL on non-NVIDIA hardware or want to decouple your physics engine from your accelerator budget. Probably overkill if you’re already all-in on CUDA and Isaac Gym.