Loading Events

« All Events

  • This event has passed.

Statistics Seminar

February 3 @ 4:00 pm - 5:00 pm

Title: Demystifying (Deep) Reinforcement Learning: The Optimist, The Pessimist, and Their Provable Efficiency  

Presenter: Zhuoran Yang Princeton University

Abstract: Coupled with powerful function approximators such as deep neural networks, reinforcement learning (RL) achieves tremendous empirical successes. However, its theoretical understandings lag behindIn particular, it remains unclear how to provably attain the optimal policy with a finite regret or sample complexity. In this talk, we will present the two sides of the same coin, which demonstrates an intriguing duality between optimism and pessimism. – In the online setting, we aim to learn the optimal policy by actively interacting with an environment. To strike a balance between exploration and exploitation, we propose an optimistic least-squares value iteration algorithm, which achieves a \sqrt{T} regret in the presence of linear, kernel, and neural function approximators. – In the offline setting, we aim to learn the optimal policy based on a dataset collected a priori. Due to a lack of active interactions with the environment, we suffer from the insufficient coverage of the dataset. To maximally exploit the dataset, we propose a pessimistic least-squares value iteration algorithm, which achieves a minimax-optimal sample complexity. 

Link: https://uwmadison.zoom.us/j/95890517940

Details

Date:
February 3
Time:
4:00 pm - 5:00 pm