Taming elusive policies across PyTorch, TensorFlow, and MindSpore

What it does XuanCe is a modular deep reinforcement learning library that implements a wide catalog of single-agent, multi-agent, model-based, and contrastive RL algorithms. It wraps environments ranging from CartPole and MuJoCo to StarCraft II and Google Research Football in one codebase, so you can switch between DQN, MAPPO, or DreamerV3 without switching repos.

The interesting bit The project’s name translates roughly to “elusive policy,” an honest admission that deep RL is notoriously sensitive to hyperparameters and unstable training. Rather than pretending otherwise, the library offers automatic hyperparameter tuning and parallel environments as built-in safeguards. Its more unusual twist is trilingual backend support: the library explicitly targets PyTorch, TensorFlow 2, and MindSpore, which is rare in a field where most libraries pick one framework and dig in.

Key highlights

• Covers single-agent (PPO, SAC, TD3), multi-agent (QMIX, MADDPG, MAPPO), model-based (DreamerV2, DreamerV3), and contrastive (CURL, SPR, DrQ) methods.
• Targets cross-platform use—Linux, Windows, and macOS—with CPU, GPU, and multi-GPU distributed training.
• Supports automatic hyperparameter tuning and visualization via TensorBoard or Weights & Biases.
• Publishes benchmark results in a separate repository and maintains full documentation in both English and Chinese.

Verdict Researchers who need to prototype across single-agent, multi-agent, and model-based methods without switching libraries should start here. If you are already locked into one framework and only need a single algorithm, it may be more toolbox than you need.