rlpyt provides modular, optimized implementations of common deep RL algorithms in PyTorch, supporting all three major families of model-free methods: policy gradient, deep Q-learning, and Q-function policy gradient. The library offers unified infrastructure for running experiments serially or in parallel, with optional multi-GPU optimization using PyTorch’s DistributedDataParallel. It supports CPU or GPU training, synchronous or asynchronous sampling, and full recurrent agent capabilities for small- to medium-scale research.