Mario for RL, now with an itemized reward receipt
The classic NES Mario environments, rebuilt for modern Gymnasium, with reward shaping you can audit instead of reverse-engineer.

What it does
Wraps the nes-py NES emulator as Gymnasium environments for four Mario titles: Super Mario Bros., SMB2 (Lost Levels), SMB2 (USA), and SMB3. Full-game environments give an agent three lives to work through the game’s 32 stages; single-stage variants (IDs like SuperMarioBros-4-2-v0) hand over one life and one level. Every step returns a dense info dict — position, coins, score, clock, powerup status, cross-game progress, death, and task_id — and a bundled CLI lets you play by keyboard or watch an agent flail at random. It targets Gymnasium’s modern reset, step, render-mode, and truncation semantics, and requires Python 3.13 or newer.
The interesting bit
The reward function is the quiet star. Progress pays out only when the agent reaches a new best position, so backtracking costs nothing, and event terms — coins, cherries, powerups, deaths, time pressure — are added only where the game’s RAM exposes reliable counters. The total is clipped to (-15, 15), but info also carries reward_components, reward_total_unclipped, and reward_total_clipped, so you can log or re-shape rewards without redoing the RAM-dependent math. Cut-scenes never reach the agent either: they’re skipped by hacking NES RAM, and when one can’t be skipped, the environment simply locks the process until the emulator is ready — refreshingly honest jank.
Key highlights
- Four titles, each as a full-game environment plus per-stage variants down to individual world/stage IDs.
- Reward instrumentation: per-step components plus unclipped and clipped totals in
info. - Task metadata API —
all_tasks,task_ids,smb3_stage_matrix— for curricula and evaluation matrices without constructing ROM-backed environments. - Action-space presets (
RIGHT_ONLY,SIMPLE_MOVEMENT,COMPLEX_MOVEMENT) to tame the default 256-action NES button space. - Registered with Gymnasium’s
TimeLimitat 9999999 steps, preserving the historical cap while game logic ends episodes withterminated=True.
Caveats
- The native
nes-pydependency is the usual suspect: LinuxGLIBCXXloader errors and Windows compiler-toolchain failures typically trace to the emulator, not this wrapper. - SMB3 single-stage support is thin — only
1-1,1-2,1-4, and1-6are validated entry points; the stage matrix catalogs more courses than are actually registered. - Code is MIT, but the Nintendo ROMs are explicitly excluded from that license, and the project is unaffiliated with Nintendo.
Verdict If you do reinforcement learning and want a familiar benchmark whose reward function you can actually inspect — or you’re building multi-task curricula over classic games — this is the one to reach for. If you need something legally unambiguous or can’t stomach a native emulator dependency, keep walking.
Frequently asked
- What is Kautenja/gym-super-mario-bros?
- The classic NES Mario environments, rebuilt for modern Gymnasium, with reward shaping you can audit instead of reverse-engineer.
- Is gym-super-mario-bros open source?
- Yes — Kautenja/gym-super-mario-bros is open source, released under the MIT license.
- What language is gym-super-mario-bros written in?
- Kautenja/gym-super-mario-bros is primarily written in Python.
- How popular is gym-super-mario-bros?
- Kautenja/gym-super-mario-bros has 910 stars on GitHub.
- Where can I find gym-super-mario-bros?
- Kautenja/gym-super-mario-bros is on GitHub at https://github.com/Kautenja/gym-super-mario-bros.