Lectures and Video from DeepLearn 2022

Link to Slides:
Link to Video:
Part 1, Goals and Challenges and solutions!

Introduction to the ODE method in a simple deterministic setting, with applications to extremum seeking control (a class of algorithms for online optimization, with applications to reinforcement learning).

Much is taken from Chapter 4, and 2022 publications available on arXiv, such as

Part 2, Variance Matters

The theoretical side of reinforcement learning has focused almost entirely on stochastic models for algorithm design and analysis.   This talk surveys techniques for algorithm design and testing, building on part 1.

The material is taken from Chapter 8,  and recent tutorials and articles including

Part 3, TD and Q-Learning

Covers final two chapters: All about algorithm design for TD- and Q-learning in a stochastic environment.

Much of Part II of CS&RL is based on handouts created over the years, some of which evolved to become