Beijing Institute of Mathematical Sciences and Applications

Computational & Applied Mathematics Group

Mo Zhou

Thursday, November 16, 2023 11:30 AM - 1:30 PM

Online

Tencent 677 1805 8331 ()

Solving the stochastic optimal control problem and its associated Hamilton—Jacobi—Bellman (HJB) equation poses significant challenges due to complexity and non-convexity. In this presentation, we introduce an innovative actor-critic approach tailored to address this complexity. Our method involves deriving an explicit derivative for the cost functional and implementing a policy gradient method for the actor (control) update. The necessity of the current control's value function prompts the development of a policy evaluation process for the critic. We present compelling numerical evidence demonstrating the efficacy of our algorithm and provide rigorous proofs of exponential convergence rates for both the actor and the critic under mild assumptions. Furthermore, we establish a convergence rate for the joint actor-critic dynamics within a single time scale, showcasing the robustness and efficiency of our proposed framework.

Mo Zhou (周默) is an assistant adjunct Professor at UCLA, where he conducts cutting-edge research at the intersection of optimal control, mean-field game problems and deep learning. Currently, he is in Prof. Stan Osher's and Prof. Hayden Schaeffer's research groups. Before joining UCLA, Mo earned his Ph.D. at Duke University, where he was mentored by Prof. Jianfeng Lu. Prior to that, he was an undergraduate at Tsinghua University.