What it does NeuralKart plays Mario Kart 64 in real-time through the BizHawk emulator, using a convolutional neural network to steer from raw pixels. The twist: it was trained via DAGGER, an iterative imitation-learning algorithm, but the “expert” it imitates isn’t a human—it’s an offline search AI that reads memory locations to plan perfect trajectories. The CNN never gets that privileged view at test time; it has to generalize from pixels alone.

The interesting bit The project uses the clipboard to ferry frames from Lua inside BizHawk to a Python prediction server. It’s a hack, and the README admits it. The training pipeline also requires hand-editing a path to train.py on line 90 of a Lua script, which gives the whole thing a charmingly research-grade feel.

Key highlights

• Three-stage pipeline: search AI generates recordings → CNN trains on those frames → DAGGER loop alternates playing and retraining
• Pre-trained weights and save states provided for specific tracks; not all tracks are supported
• Built for BizHawk 1.12.2, Python 3, and Windows only
• Derived from TensorKart, with the search-based teacher idea borrowed from a NIPS paper on Atari
• Includes a live demo mode where you can hit arrow keys to nudge the AI and test its stability

Caveats

• Windows-only, pinned to an old BizHawk version, and the clipboard IPC is fragile enough that the README warns you not to copy-paste while it’s running
• The iterative improvement loop requires manual code edits to even launch

Verdict Worth a look if you’re studying imitation learning or want a concrete, slightly messy example of DAGGER in a game environment. Skip it if you want something that runs out of the box on modern hardware or non-Windows systems.