Link to Slides:

Link to Video:

Introduction to the ODE method in a simple deterministic setting, with applications to extremum seeking control (a class of algorithms for online optimization, with applications to reinforcement learning).

Much is taken from Chapter 4, and 2022 publications available on arXiv, such as

• Markovian Foundations for Quasi-Stochastic Approximation with Applications to Extremum Seeking Control

The theoretical side of reinforcement learning has focused almost entirely on stochastic models for algorithm design and analysis.   This talk surveys techniques for algorithm design and testing, building on part 1.

The material is taken from Chapter 8,  and recent tutorials and articles including

• The ODE Method for Asymptotic Statistics in Stochastic Approximation and Reinforcement Learning

Covers final two chapters: All about algorithm design for TD- and Q-learning in a stochastic environment.