Teaching RL to write stock-picking formulas

What it does AlphaGen automates the discovery of formulaic stock factors—think “rank(volume, 20) / close”—using reinforcement learning. It trains an agent to build symbolic expressions that predict returns, then combines the best ones into weighted pools. The result is human-readable alphas, not opaque embeddings.

The interesting bit The project treats alpha generation as a tree-structured RL problem: the agent picks operators and operands via PPO, with an LSTM feature extractor guiding the policy. It also ships an LLM-based generator (alphagen_llm) that iteratively prompts a language model to produce and refine alphas—useful if you want to compare RL against a chatbot’s intuition.

Key highlights

• KDD 2023 Applied Data Science track; code reproduces the paper’s experiments.
• Built-in pipeline uses Microsoft’s Qlib for metadata but pulls actual price/volume data from baostock (the authors distrust Qlib’s data timeliness).
• AlphaCalculator interface lets you plug in your own backtest engine without touching the RL core.
• Includes modified baselines: gplearn (genetic programming) and DSO (deep symbolic regression) for head-to-head comparison.
• Experimental trading demos via Qlib backtesting in backtest.py and trade_decision.py.

Caveats

• The README warns that alpha values can have “drastically different scales” and recommends normalization before pooling—suggesting the raw RL outputs aren’t immediately portfolio-ready.
• Trading support is explicitly labeled “Experimental”; this is a factor research tool, not a turnkey trading system.
• Data setup is split across two sources (Qlib metadata + baostock prices), which adds friction.

Verdict Worth a look if you’re a quant researcher who wants interpretable factors or needs a baseline for comparing RL, GP, and LLM-based discovery. Skip it if you need production execution or a single-button data pipeline.