By Shaofeng Zou (University of Buffalo)

Talk Abstract: In this talk, I will present our recent work on robust reinforcement learning (RL) under model uncertainty with algorithm design, convergence and complexity analysis and experimental results. RL recently has achieved great success in various benchmark tasks, e.g., beating human champions in the game of Go and achieving grandmaster level in video games. However, existing RL approaches usually assume that a learned policy will be deployed in the same environment as the one it was trained in. Such an assumption is often violated in practice, e.g., a robot trained in experimental environment will be deployed in real outdoor environment, and a policy will be deployed in an adversarial environment under potential attacks, which could lead to a significant performance degradation. In this talk, we first review results of model-based approaches for robust RL under model uncertainty (discounted reward setting). We will then present our recent results on the design and analysis for online model-free robust RL under adversarial state transition perturbation, including value-based method and policy gradient method. We will further introduce our results on robust RL under the fundamental average-reward setting, which finds applications in systems that operate for an extended period of time. We develop a fundamental framework for the robust average-reward setting with a comprehensive design and analysis for both the model-based and model-free approaches.

Speaker Bio: Shaofeng Zou is an Assistant Professor, at the Department of Electrical Engineering, University at Buffalo, the State University of New York. He received the Ph.D. degree in Electrical and Computer Engineering from Syracuse University in 2016. He received the B.E. degree (with honors) from Shanghai Jiao Tong University, Shanghai, China, in 2011. He was a postdoctoral research associate at the Coordinated Science Lab, University of Illinois at Urbana-Champaign during 2016-2018. Dr. Zou’s research interests include reinforcement learning, machine learning, statistical signal processing and information theory. Dr. Zou’s research is supported by several projects from NSF.