Unreal Engine 4 as your gym teacher

What it does Holodeck wraps Unreal Engine 4 into an OpenAI Gym-style Python interface for training RL agents. You get UAVs wandering cities, multi-agent scenarios, and sensor dictionaries back as numpy arrays — pip install and go.

The interesting bit The “batteries included” pitch actually holds up: seven pre-built worlds, headless GPU-accelerated rendering, and a persistent-action model for multi-agent control where act sets a command and tick advances time. That’s a thoughtful split for coordination problems.

Key highlights

• Gym-like API: make(), reset(), step() — no Unreal Blueprints required
• Multi-agent via act() + tick() with persistent commands per agent
• Headless mode keeps GPU acceleration; up to 2× real-time simulation claimed
• 7+ worlds with scenario configs; sensor data returned as named numpy arrays
• Linux and Windows; Python ≥3.5

Caveats

• The 2× real-time speed claim lacks benchmarks or conditions in the README
• BYU lab project with 596 stars; unclear how actively maintained (build badge points to a Jenkins instance, not GitHub Actions)

Verdict Worth a spin if you need prettier visuals than MuJoCo or PyBullet but don’t want to wrestle UE4 directly. Skip if you need battle-tested community scale or deterministic physics guarantees.