Bipedal RL That Separates Gait from Gym Hacks

What it does

This repository is the companion code for three research papers on bipedal locomotion. It trains simulated humanoid robots—specifically the JVRC and Unitree H1 models—to walk, stand, and follow planned footsteps inside MuJoCo using reinforcement learning. A minimal cartpole swing-up environment is also included as a pipeline sanity check.

The interesting bit

Rather than burying reward logic inside monolithic environment code, the authors split it into explicit, parameterised functions in tasks/rewards.py that are easier to test and tune. The gait machinery borrows clock-based reward terms from OSU DRL’s Cassie work, so the policies are encouraged to learn rhythmic, periodic walking instead of merely stumbling toward a goal.

Key highlights

• Implements three distinct IEEE/arxiv papers: omnidirectional walking on uneven terrain, walking with simulated current feedback, and footstep-following gaits.
• BaseHumanoidEnv and BaseTask split robot simulation from task logic, while tasks/rewards.py keeps reward weights in configurable YAML files.
• Ships with ready-made MuJoCo models and environments for JVRC and Unitree H1, plus a documented inheritance pattern for adding new hardware.
• Includes a cartpole swing-up task to validate the RL pipeline without waiting for a humanoid to learn to stand.

Verdict

Grab it if you are a robotics researcher looking to bootstrap RL locomotion for a new humanoid in MuJoCo—the abstractions are built for extension. Pass if you need a general-purpose RL framework or drop-in hardware deployment; this is tightly coupled to bipedal walking research.