MushroomRL is a modular Python library for reinforcement learning that supports both classical algorithms (Q-Learning, SARSA, FQI) and deep RL algorithms (DQN, DDPG, SAC, TD3, TRPO, PPO). It integrates with popular ML frameworks (PyTorch, TensorFlow) and RL benchmarks (Gymnasium, PyBullet, MuJoCo, DeepMind Control Suite). The library allows researchers to perform RL experiments in a straightforward manner.