Beijing Institute of Mathematical Sciences and Applications

This course introduces the fundamental theory and applications of Optimal Control. The primary objective is to equip students with the tools to analyze and solve problems where dynamic systems must be operated in an optimal manner. The curriculum is unified by the Dynamic Programming (DP) approach pioneered by Richard Bellman. We will explore how the Bellman Equation (or Hamilton-Jacobi-Bellman equation) provides sufficient conditions for optimality. The course is divided into two halves: the first covers discrete-time systems, including deterministic, stochastic, and Markov Control Processes (MCPs) over finite and infinite horizons. The second half transitions to continuous-time systems, covering ordinary differential equations, general MCPs, and diffusion processes. By the end, students will understand the advantages of DP, including its role in generating feedback controls and its connection to approximation algorithms like value and policy iteration, which are foundational to modern fields like reinforcement learning.