Flow control for RL: 61 environments, six solvers, one API

What it does

HydroGym is a Python platform that turns computational fluid dynamics simulations into standard reinforcement-learning environments. It wraps solvers such as Firedrake, NEK5000, MAIA, and JAX-Fluids behind a Gymnasium-compatible interface, so researchers can plug in agents from Stable-Baselines3 or RLlib without touching the underlying CFD code. The environments range from canonical benchmarks—cylinder wakes, cavity flows, and NACA0012 airfoils—to high-Reynolds turbulence and fully differentiable compressible flows.

The interesting bit

The real trick is abstraction without homogenization: HydroGym keeps the distinct personalities of six very different backends—finite element, lattice Boltzmann, spectral element—while presenting them through the same FlowEnv interface. That means you can benchmark an agent on a cheap 2D Firedrake cylinder wake, then zero-shot transfer it to a 3D NEK5000 turbulent channel without rewriting your training loop.

Key highlights

• 61+ environments across 6 solver backends, including FEM, LBM, spectral element, finite volume, and JAX-based differentiable solvers.
• Native Gymnasium API compatibility with standard RL libraries like Stable-Baselines3 and RLlib.
• Scales from local GPU workstations to MPI-parallelized HPC deployments.
• Covers Reynolds numbers up to 400,000, including 3D turbulent boundary layers and compressible shock-vector control.
• Built-in observation types: force sensors, velocity/pressure probes, vorticity, and stress.

Caveats

• Environment checkpoints are hosted on HuggingFace and require an internet connection for initial download; air-gapped HPC nodes are not supported out of the box.
• The maintainers strongly recommend Docker, suggesting native dependency management across the six solver backends is brittle.

Verdict

Researchers in flow control or RL-for-PDEs should start here; if your fluids problem fits a standard benchmark, this saves months of CFD plumbing. Skip it if you need custom geometries or non-Gymnasium control loops, since the value is in the standardized interface rather than the solvers themselves.