BIMSA >
Tsinghua-BIMSA Computational & Applied Mathematics (CAM) Seminar
Advancing Stochastic Optimal Control: An Actor-Critic Framework
Advancing Stochastic Optimal Control: An Actor-Critic Framework
Organizer
Computational & Applied Mathematics Group
Speaker
Mo Zhou
Time
Thursday, November 16, 2023 11:30 AM - 1:30 PM
Venue
Online
Online
Tencent 677 1805 8331
()
Abstract
Solving the stochastic optimal control problem and its associated Hamilton—Jacobi—Bellman (HJB) equation poses significant challenges due to complexity and non-convexity. In this presentation, we introduce an innovative actor-critic approach tailored to address this complexity. Our method involves deriving an explicit derivative for the cost functional and implementing a policy gradient method for the actor (control) update. The necessity of the current control's value function prompts the development of a policy evaluation process for the critic. We present compelling numerical evidence demonstrating the efficacy of our algorithm and provide rigorous proofs of exponential convergence rates for both the actor and the critic under mild assumptions. Furthermore, we establish a convergence rate for the joint actor-critic dynamics within a single time scale, showcasing the robustness and efficiency of our proposed framework.
Speaker Intro
Mo Zhou (周默) is an assistant adjunct Professor at UCLA, where he conducts cutting-edge research at the intersection of optimal control, mean-field game problems and deep learning. Currently, he is in Prof. Stan Osher's and Prof. Hayden Schaeffer's research groups. Before joining UCLA, Mo earned his Ph.D. at Duke University, where he was mentored by Prof. Jianfeng Lu. Prior to that, he was an undergraduate at Tsinghua University.