Training agents to trade crypto, then trading up to TensorTrade

What it does RLTrader is an OpenAI Gym-compatible environment for cryptocurrency markets. It wraps the stable-baselines PPO2 implementation with data ingestion, hyperparameter search, and train-test splitting to see whether an RL agent generalizes to unseen price data. A CLI ties the pieces together and stores optimization trials in an SQLite database.

The interesting bit The README explicitly admits that default PPO2 parameters will likely lose money, so the project includes a dedicated optimization phase—drawing on feature engineering, statistical modeling, and Bayesian optimization, per the authors’ write-ups—before agents are trained with the best found settings. It also ships with a live trading visualization.

Key highlights

• Built on OpenAI Gym and stable-baselines PPO2
• Hyperparameter optimization pipeline (optimize.py) with SQLite persistence
• Trains on historical data, then tests on completely new data for generalization
• Includes live trading visualization
• Supports Docker, Vagrant, and Google Colab environments

Caveats

• Development has slowed in favor of TensorTrade, so this is effectively a legacy predecessor
• Hyperparameter optimization can take hours to days depending on hardware
• README is heavy on environment setup and light on architectural or performance details

Verdict A solid reference if you want to see how an end-to-end RL crypto pipeline fits together. If you need something actively maintained, the authors direct you to TensorTrade instead.