Modern control theory
This course explores the deep connections between optimal control and reinforcement learning, bridging classical techniques (Dynamic Programming, LQR, MPC) with modern data-driven methods (Q-Learning, Policy Gradient, Deep RL). Students will learn: Mathematical foundations (Bellman equations, value/policy iteration); Optimal control (LQR, LQG, Model Predictive Control); Approximate DP & RL (Monte Carlo, TD Learning, Actor-Critic methods); Applications in robotics, autonomous systems, and finance. The course balances theory (convergence, stability) and implementation (Python examples).
Lecturer
Date
3rd September ~ 31st December, 2025
Location
| Weekday | Time | Venue | Online | ID | Password |
|---|---|---|---|---|---|
| Wednesday | 14:20 - 16:55 | A3-1-101 | ZOOM 03 | 242 742 6089 | BIMSA |
Prerequisite
Linear algebra, probabilistic theory, calculus, optimization
Syllabus
Foundations of Optimal Control & Exact DP
1: Introduction to Dynamic Programming
2: Deterministic Continuous-Time Optimal Control
3: Stochastic DP and the LQG Problem
4: Model Predictive Control (MPC)
5: Infinite Horizon Problems
6: Shortest Path Problems & Computational Methods
Approximate DP & RL Basics
7: Approximate Value Iteration
8: Monte Carlo & Temporal Difference Learning
9: Policy Gradient Methods
10: Approximate Policy Iteration
Advanced Topics
11: Robust DP and H infinity Control
12: Multiagent RL and Games
13: Inverse Reinforcement Learning
14: Deep Reinforcement Learning
1: Introduction to Dynamic Programming
2: Deterministic Continuous-Time Optimal Control
3: Stochastic DP and the LQG Problem
4: Model Predictive Control (MPC)
5: Infinite Horizon Problems
6: Shortest Path Problems & Computational Methods
Approximate DP & RL Basics
7: Approximate Value Iteration
8: Monte Carlo & Temporal Difference Learning
9: Policy Gradient Methods
10: Approximate Policy Iteration
Advanced Topics
11: Robust DP and H infinity Control
12: Multiagent RL and Games
13: Inverse Reinforcement Learning
14: Deep Reinforcement Learning
Reference
Bertsekas, D. P. (2019). Reinforcement Learning and Optimal Control. Athena Scientific.
Sutton & Barto, Reinforcement Learning: An Introduction
Bruno C. da Silva, Reinforcement Learning Lectures Notes (Fall 2022)
Sutton & Barto, Reinforcement Learning: An Introduction
Bruno C. da Silva, Reinforcement Learning Lectures Notes (Fall 2022)
Audience
Undergraduate
, Advanced Undergraduate
, Graduate
, Postdoc
, Researcher
Video Public
No
Notes Public
No
Language
Chinese
, English
Lecturer Intro
Jiao Xiaopei graduated with a bachelor's degree from the Zhi Yuan College of Shanghai Jiao Tong University (Physics Department) in 2017 and obtained his PhD from the Department of Mathematical Sciences at Tsinghua University in 2022, under the guidance of Professor Stephen Shing-Toung Yau (IEEE Fellow, former tenured professor at the University of Illinois at Chicago). He has conducted postdoctoral research at the Beijing Institute of Mathematica Science and Application and at the University of Twente in the Netherlands (under the guidance of Professor Johannes Schmidt-Hieber, Fellow of the Institute of Mathematical Statistics). His current research interests include control theory, numerical partial differential equations, and bioinformatics.