Uber's genetic algorithms for Atari, no backprop required

What it does

This repo packages Uber AI Labs’ distributed implementations of Deep Genetic Algorithm (DeepGA), Evolution Strategies (ES), and novelty-seeking variants (NS-ES, NSR-ES) for reinforcement learning. It runs on AWS or locally via Redis-backed workers, targeting Atari games and MuJoCo humanoid tasks. A bundled visualization tool called VINE lets you inspect population dynamics interactively.

The interesting bit

The core pitch: genetic algorithms can be a “competitive alternative” to backprop for training deep nets on RL problems — a claim that still raises eyebrows in gradient-worshipping circles. The code is built atop OpenAI’s evolution-strategies-starter, extended with novelty-seeking exploration tricks and a GPU-accelerated implementation folder.

Key highlights

• Implements five algorithms: ES, NS-ES, NSR-ES, DeepGA, and random search baseline
• Includes VINE (Visual Inspector for NeuroEvolution) for interactive population visualization
• GPU implementation folder for “more efficient” GPU usage (details in sub-README)
• Docker Compose setup for containerized runs
• Pre-built configs for Frostbite and Humanoid-v1 environments

Caveats

• MuJoCo experiments require your own license and binary; not included
• README has a typo in “Neurevolution” that has apparently survived since 2017
• GPU implementation details are offloaded to a separate README; unclear how mature it is

Verdict

Worth a spin if you’re researching neuroevolution or need a gradient-free baseline for RL comparisons. Skip it if you want maintained, batteries-included code — this is research artifact territory, and the OpenAI upstream has likely moved on.