The canonical RL course, now with copy-pasteable Keras code

What it does This repository is a study companion to David Silver’s 10-week UCL reinforcement learning course. It collects notes and provides Jupyter implementations of the algorithms discussed—spanning Markov decision processes to policy gradients—using Keras with a TensorFlow backend and OpenAI Gym environments.

The interesting bit Instead of just summarizing slides, it makes the lecture math executable week by week, acting as a debugging-friendly bridge between Silver’s theory and practice. The repo also openly invites ports to PyTorch or Caffe, framing itself as a learning scaffold rather than a production framework.

Key highlights

• Covers the full 10-week syllabus: MDPs, dynamic programming, model-free prediction and control, value function approximation, policy gradients, and exploration vs. exploitation.
• Implements algorithms in Keras against OpenAI Gym environments, with dependencies limited to TensorFlow, Keras, NumPy, and Gym.
• Explicitly references and builds on prior community work, notably Denny Britz’s reinforcement-learning repo.
• MIT-licensed and open to contributions porting implementations to other frameworks.

Caveats

• The stack is dated: Keras with TensorFlow backend and an open call for Caffe contributions place this firmly in the mid-2010s deep learning era.
• A Travis CI badge in the README still points to an unrelated repository (athityakumar/colorls), suggesting maintenance has been minimal.

Verdict Best for students working through Silver’s lectures who need runnable reference code. Skip it if you’re after a modern, maintained RL library or native PyTorch examples.