Advancing Stochastic Optimal Control: An Actor-Critic Framework
组织者
Computational & Applied Mathematics Group
演讲者
周默
时间
2023年11月16日 11:30 至 13:30
地点
Online
线上
Tencent 677 1805 8331
()
摘要
Solving the stochastic optimal control problem and its associated Hamilton—Jacobi—Bellman (HJB) equation poses significant challenges due to complexity and non-convexity. In this presentation, we introduce an innovative actor-critic approach tailored to address this complexity. Our method involves deriving an explicit derivative for the cost functional and implementing a policy gradient method for the actor (control) update. The necessity of the current control's value function prompts the development of a policy evaluation process for the critic. We present compelling numerical evidence demonstrating the efficacy of our algorithm and provide rigorous proofs of exponential convergence rates for both the actor and the critic under mild assumptions. Furthermore, we establish a convergence rate for the joint actor-critic dynamics within a single time scale, showcasing the robustness and efficiency of our proposed framework.
演讲者介绍
Mo Zhou (周默) is an assistant adjunct Professor at UCLA, where he conducts cutting-edge research at the intersection of optimal control, mean-field game problems and deep learning. Currently, he is in Prof. Stan Osher's and Prof. Hayden Schaeffer's research groups. Before joining UCLA, Mo earned his Ph.D. at Duke University, where he was mentored by Prof. Jianfeng Lu. Prior to that, he was an undergraduate at Tsinghua University.