Diffusion Models as Trajectory Planners

What it does

Diffuser trains diffusion models on offline trajectory data, then generates full state-action sequences by iterative denoising. A learned value function guides the sampling process toward high-reward plans. The repo also includes branches for block-stacking and goal-conditioned maze navigation via trajectory inpainting.

The interesting bit

Instead of outputting actions one by one, the model generates an entire future path at once. Planning becomes conditional sampling: start with random noise and refine it into a coherent hopper gait or walker stride. The maze2d branch extends this to inpainting, filling in the path between fixed start and goal states.

Key highlights

• Competitive results on D4RL locomotion, including an average score of 77.5 across HalfCheetah, Hopper, and Walker2d tasks, with individual runs hitting 103–107 on expert datasets.
• Pretrained model weights and evaluation logs are provided for direct comparison.
• The model has been upstreamed into the Hugging Face diffusers library.
• Separate branches cover block-stacking and maze navigation, though the main README focuses on locomotion.

Caveats

• The main branch is tightly coupled to D4RL locomotion; block-stacking and maze experiments live in side branches with minimal documentation in the main README.
• A value-function scaling bug was patched in October 2022, so forks or prior checkpoints may need scrutiny.
• Planning hyperparameters are loaded via deferred f-strings that resolve file paths rather than altering logic directly, which can be unintuitive.

Verdict

Worth a look if you research offline RL or generative planning, especially with diffusion. Skip it if you need a plug-and-play policy library for production environments.