Teaching AIs to drive by playing TrackMania at full speed

What it does

tmrl is a Python framework for training reinforcement-learning policies in real-time environments—robots, high-frequency control systems, or video games. It ships with a complete, ready-to-run pipeline for TrackMania 2020: the AI learns to race from either raw screenshots or simplified “LIDAR” rangefinder observations, outputting analog steering, gas, and brake commands through a virtual gamepad. Under the hood it runs Soft Actor-Critic (SAC) or REDQ, with a server/client architecture that lets you collect gameplay data locally and train on remote hardware.

The interesting bit

The project treats TrackMania as a visual simulator for autonomous driving—a proxy world where crashing costs nothing and physics are repeatable. The framework’s real trick is rtgym, a spin-off library that wrangles Gymnasium environments into hard real-time constraints, which is where the actual robotics value lives. The TrackMania demo is the sugar that makes the distributed-RL medicine go down.

Key highlights

• Vision-based and LIDAR observation spaces; CNN for images, MLP for rangefinder data
• Analog control via virtual gamepad (vgamepad), another extracted sub-project
• Distributed training: arbitrary local workers, remote server, with optional TLS via tlspyo
• Hosts the TrackMania Roborace League, an ongoing AI competition
• Presented at Ubisoft Montreal as a model for using games as driving simulators

Caveats

• Default networking uses unencrypted TCP; the README repeatedly warns this is a security breach on public networks and requires manual TLS setup
• The “getting started” path is Windows-first, with Linux as a secondary track
• Pre-trained policies are competent but not superhuman: the LIDAR agent ran ~45.5s vs. a ~32s world-champion target on the test track

Verdict

Worth a look if you’re building RL for physical robots and need a real-time-safe framework with a fun, visual debugging environment. If you just want a Gymnasium wrapper for TrackMania, that’s here too—but the broader robotics tooling is the actual payload.