Reinforced Processes: Urns and Walks
$\textbf{Schedule update: The course has been revised to six teaching weeks and will conclude on 4 November 2026.}$
$\textbf{This edition covers urn models and reinforced random walks; the originally announced bandit component will not be offered.}$
This non-credit course studies urn models and reinforced random walks, with emphasis on how past observations change future transition probabilities. The course comprises twelve meetings over six teaching weeks, each consisting of two 45-minute periods.
We develop complete proofs for selected results and use them to introduce martingales, exchangeability, exponential embeddings, Lyapunov functions, and random-environment representations. The walk lectures also connect local-time profiles with branching processes. More advanced results are stated with their main ideas and references.
$\textbf{This edition covers urn models and reinforced random walks; the originally announced bandit component will not be offered.}$
This non-credit course studies urn models and reinforced random walks, with emphasis on how past observations change future transition probabilities. The course comprises twelve meetings over six teaching weeks, each consisting of two 45-minute periods.
We develop complete proofs for selected results and use them to introduce martingales, exchangeability, exponential embeddings, Lyapunov functions, and random-environment representations. The walk lectures also connect local-time profiles with branching processes. More advanced results are stated with their main ideas and references.
Lecturer
Date
21st September ~ 4th November, 2026
Location
| Weekday | Time | Venue | Online | ID | Password |
|---|---|---|---|---|---|
| Monday,Wednesday | 15:20 - 16:55 | A3-1-301 | ZOOM 04 | 482 240 1589 | BIMSA |
Prerequisite
A solid undergraduate course in probability is recommended. Familiarity with conditional expectation, Markov chains, and basic convergence concepts is helpful. Martingale method needed in the course will be reviewed or introduced as they arise. The course is intended primarily for senior undergraduate students and beginning graduate students interested in probability and stochastic processes.
Syllabus
1. Urn Models (Weeks 1–3): Classical Pólya urns, exchangeability and random limits, generalized and nonlinear urns, stochastic approximation, and phase transitions caused by different reinforcement strengths.
2. Reinforced Random Walks (Weeks 4–6). Vertex and edge reinforcement; exact urn reductions on small graphs; trapping and localization; continuous-time embeddings and occupation profiles; entropy methods on a triangle; partial exchangeability and random environments for linear edge reinforcement; branching-process and spatial Markov descriptions of one-dimensional local times.
2. Reinforced Random Walks (Weeks 4–6). Vertex and edge reinforcement; exact urn reductions on small graphs; trapping and localization; continuous-time embeddings and occupation profiles; entropy methods on a triangle; partial exchangeability and random environments for linear edge reinforcement; branching-process and spatial Markov descriptions of one-dimensional local times.
Reference
R. Pemantle, A Survey of Random Processes with Reinforcement, Probability Surveys 4 (2007), 1–79.
D. Williams. Probability with Martingales. Cambridge University Press, 1991.
H. M. Mahmoud. Pólya Urn Models, first edition. Chapman & Hall/CRC, 2008.
Additional notes and selected research articles will be provided by the instructor.
D. Williams. Probability with Martingales. Cambridge University Press, 1991.
H. M. Mahmoud. Pólya Urn Models, first edition. Chapman & Hall/CRC, 2008.
Additional notes and selected research articles will be provided by the instructor.
Audience
Advanced Undergraduate
, Graduate
Video Public
Yes
Notes Public
Yes
Language
Chinese
Lecturer Intro
Shuo Qin has been the first Chern Instructor at BIMSA. He obtained a Ph.D. in mathematics in 2024 from New York University under the supervision of Prof. Pierre Tarrès. His work is in probability theory, especially in random processes with memory or reinforcement.