A live MOBA becomes a competitive reinforcement learning arena

What it does

hok_env is an SDK and training framework that lets reinforcement learning algorithms control heroes inside Honor of Kings, a major commercial MOBA. It wraps the Windows-based gamecore with a Python interface, supports 1v1 and 3v3 modes, and ships with a reference PPO implementation. The project is explicitly designed to study generalization: agents must adapt to different heroes and opponents rather than memorizing a single matchup.

The interesting bit

Most RL benchmarks are sanitized toy worlds; this one drops agents into a live, high-fidelity 3D engine with 19 distinct heroes and real-time competitive pressure. By treating the environment as a generalization challenge rather than a fixed task, it forces researchers to solve transfer learning in the kind of messy, high-dimensional domain that real games actually are.

Key highlights

• Nineteen playable heroes, from mages to marksmen, each with unique mechanics.
• Includes a reference PPO implementation and an RL training framework aimed at cluster-scale experiments.
• Supports both 1v1 lane duels and 3v3 team skirmishes.
• Accepted to the NeurIPS 2022 Datasets and Benchmarks track.
• The SDK runs on Linux and communicates with the Windows gamecore via a ZMQ-based Python interface.

Caveats

• The gamecore is Windows-only; Linux users need Docker, Wine, or WSL2 workarounds, and a native Linux server is still listed as a future release.
• You must apply for a license and download the proprietary gamecore separately from Tencent’s AI Arena portal.
• Python support is capped at version 3.9.

Verdict

Researchers studying competitive multi-agent RL and generalization in complex action spaces should look here. If you need a lightweight, cross-platform environment that runs out of the box, look elsewhere.