Letting a neural network pick MPC weights so your dog bot doesn't trip

What it does

This project is a simulation and training framework for quadruped locomotion that pairs a high-level reinforcement learning policy with a low-level convex model-predictive controller. The RL policy, trained in NVIDIA Isaac Gym, dynamically predicts the weight parameters of the MPC rather than outputting motor torques directly. The MPC layer is a Python reimplementation of MIT’s Cheetah Software, complete with a finite-state machine for gait scheduling and a C-based OSQP solver backend.

The interesting bit

The hierarchy keeps the optimizer responsible for physics constraints and contact dynamics while the neural network handles the messier task of cost-function tuning. It is essentially a bet that MPC should do what it is good at—respecting constraints—while RL learns the knobs that are painful to dial by hand.

Key highlights

• Hierarchical control: the RL policy sets MPC cost weights, and the MPC computes ground reaction forces and swing trajectories.
• Python port of MIT Cheetah Software with a clean sensor-to-motor interface designed for mainstream simulators.
• Trains vectorized environments in NVIDIA Isaac Gym on Unitree’s Aliengo model; also supports A1 and Go1.
• Includes a finite-state machine with Locomotion and RecoveryStand states and gait switching across Trot, Walk, and Bound.
• Uses a C-based OSQP solver (mpc_osqp.cc) for the convex MPC, bound to Python.

Caveats

• Sim-to-real transfer to a physical Aliengo is shown in the gallery, but the transfer code is explicitly excluded from this repository.

Verdict

A solid reference if you are researching hierarchical RL-MPC hybrids for legged robots. Skip it if you are looking for a drop-in sim-to-real pipeline or a pure end-to-end RL walking policy.