Seven ways to copy an expert, all running on SAC

What it does

This repo houses six recent imitation learning algorithms—AdRIL, DRIL, GAIL, GMMIL, PWIL, and RED—plus BC pretraining, all built on a single Soft Actor-Critic (SAC) implementation. It targets standard continuous-control benchmarks, specifically MuJoCo environments fed with D4RL “expert-v2” trajectories, and uses Hydra configs to keep experiments reproducible. The goal is to isolate the imitation strategy from the underlying RL engine so comparisons are actually fair.

The interesting bit

The authors take the “pragmatic” label seriously: instead of letting each algorithm bring its own bespoke RL backbone, they factor out SAC and expose a dense matrix of toggles—state-only learning, absorbing states, mixed expert/agent batches, multiple discriminator losses, and reward shaping for GAIL. They also ship pre-tuned hyperparameters for specific algorithm-and-trajectory-count pairs, plus Bayesian optimization support via Ax, suggesting they actually ran the experiments rather than just wrapping other people’s code.

Key highlights

• Six IL algorithms (AdRIL, DRIL, GAIL, GMMIL, PWIL, RED) plus BC pretraining, all sharing one SAC backbone
• GAIL alone supports AIRL/FAIRL rewards, gradient penalties, spectral normalization, Mixup/PUGAIL losses, and entropy bonuses
• Benchmarked on D4RL expert-v2 MuJoCo tasks with pre-tuned hyperparameters for specific trajectory counts
• Built-in hyperparameter sweeps and Bayesian optimization via Hydra and Ax
• Clean, config-driven experiment setup using YAML files

Caveats

• Benchmarking and pre-tuned hyperparameters are limited to four MuJoCo continuous-control environments (ant, halfcheetah, hopper, walker2d) using D4RL expert-v2 data
• The project is structured as a reproducibility codebase for a specific ACML 2023 paper; the interface is train.py scripts rather than a public API or library

Verdict

Continuous-control researchers who need to compare IL methods on a level playing field will find the unified SAC backbone and pre-tuned configs valuable. Those working with discrete actions, Atari, or seeking a pip-installable library should keep scrolling.