When GPT-2 dreams of pipes and Goombas

What it does MarioGPT finetunes distilgpt2 on levels from Super Mario Bros and The Lost Levels, then generates new level layouts from plain-text prompts. The output is a tile map you can view as an image, save to disk, or load into a Java-based simulator to play or test with an A* agent.

The interesting bit The authors treat level design as a language modeling problem: tokenize the tile grid, feed it to a small LM, and steer generation with prompts. It is a deliberately simple architecture (the README calls it “not perfect” but “a great first step”), which makes the results easy to hack on or retrain.

Key highlights

• Hosted on HuggingFace Hub at shyamsn97/Mario-GPT2-700-context-length; pip-installable as mario-gpt
• Supports continuation: seed generation with an existing level and extend it
• Includes a browser demo and local Java simulator for interactive play
• Training code is exposed via MarioGPTTrainer with a short notebook example
• Paper accepted at NeurIPS 2023

Caveats

• Inpainting and open-ended generation from the paper are listed in “Future Plans” but not yet implemented
• Local play requires Java 8+ installed separately
• The authors note generation is “not perfect” and recommend temperature tuning (~2.0–2.4) for playable results

Verdict Worth a spin if you are curious about procedural generation or want a concrete, small-scale example of finetuning transformers for structured output. Skip if you need production-ready level design tools; this is research code with rough edges left visible.