12 x 2 Lectures on Deep Learning, Geometry, Statistics and Statistical Mechanics
Modern theoretical approaches to deep learning draw heavily on ideas from geometry, statistical physics, and the theory of interacting dynamical systems. At the same time, many classical concepts from statistical learning theory such as generalization, bias–variance tradeoffs, regularization and kernel methods remain central to understanding neural networks. This lecture series presents a unified view of these perspectives. We discuss how deep learning models can be understood as high-dimensional dynamical systems, how training dynamics lead to kernel limits and mean-field descriptions, and how geometric principles such as symmetry and equivariance guide modern architectures.
The course consists of 12 weeks with two lectures per week. Each week focuses on a distinct topic and is designed to be as self-contained as possible. The first lecture provides a conceptual overview of the main theoretical ideas, while the second lecture focuses on practical implementation in JAX.
The course consists of 12 weeks with two lectures per week. Each week focuses on a distinct topic and is designed to be as self-contained as possible. The first lecture provides a conceptual overview of the main theoretical ideas, while the second lecture focuses on practical implementation in JAX.
Lecturer
Date
9th March ~ 3rd July, 2026
Location
| Weekday | Time | Venue | Online | ID | Password |
|---|---|---|---|---|---|
| Monday,Friday | 13:30 - 15:05 | A3-2a-201 | ZOOM 04 | 482 240 1589 | BIMSA |
Prerequisite
A background in geometry, statistical physics, quantum field theory, or mathematical physics, broadly construed. Alternatively, a background in deep learning and/or statistical learning theory and comfort with linear algebra, probability, and calculus.
Syllabus
The lectures are organized into 12x2, with broader organizing themes.
STATISTICAL LEARNING FOUNDATIONS
1. Statistical Learning: Learning not memorization, Bias, Variance and Regularization. Learning in high dimensions.
2. Generalized Linear Models: Regression and Classification, Over-fitting, regularization
PRIMER ON NEURAL NETWORKS
3. Multi-layer Perceptrons (Fully Connected Neural Networks): Universal Approximation, Backpropagation, Depth
4. Gradient-Based Optimization: stochastic gradients, improving gradient descent, deep learning dynamics, gradient flows
THEORY OF LARGE NEURAL NETWORKS
5. The Neural Tangent Kernel: Training dynamics under gradient descent, lazy learning, escaping lazy learning
6. Mean Field Theory of Neural Networks: Connections to Optimal Transport, no lazy learning
7. Deep Equilibrium Models: Residual Learning, Infinite depth
GEOMETRY AND SYMMETRY AS GUIDING PRINCIPLES IN DEEP LEARNING
8. Convolutional Neural Networks: Image data, Convolutions, Padding and Pooling
9. Geometric Deep Learning: Abstracting from ConvNets, Symmetry, Invariance, Equivariance
10. Deep Learning on Graphs and Sequences: Graph Neural Networks, Message Passing and Self Attention
GENERATIVE MODELS
11. Generative Models: Adversarial Networks (GANs), Wasserstein GANs, Variational Auto-Encoders
STATISTICAL PHYSICS OF DEEP LEARNING
12. Statistical Physics of Deep Learning: Mean fields revisited, Energy landscapes, phase transitions in learning
STATISTICAL LEARNING FOUNDATIONS
1. Statistical Learning: Learning not memorization, Bias, Variance and Regularization. Learning in high dimensions.
2. Generalized Linear Models: Regression and Classification, Over-fitting, regularization
PRIMER ON NEURAL NETWORKS
3. Multi-layer Perceptrons (Fully Connected Neural Networks): Universal Approximation, Backpropagation, Depth
4. Gradient-Based Optimization: stochastic gradients, improving gradient descent, deep learning dynamics, gradient flows
THEORY OF LARGE NEURAL NETWORKS
5. The Neural Tangent Kernel: Training dynamics under gradient descent, lazy learning, escaping lazy learning
6. Mean Field Theory of Neural Networks: Connections to Optimal Transport, no lazy learning
7. Deep Equilibrium Models: Residual Learning, Infinite depth
GEOMETRY AND SYMMETRY AS GUIDING PRINCIPLES IN DEEP LEARNING
8. Convolutional Neural Networks: Image data, Convolutions, Padding and Pooling
9. Geometric Deep Learning: Abstracting from ConvNets, Symmetry, Invariance, Equivariance
10. Deep Learning on Graphs and Sequences: Graph Neural Networks, Message Passing and Self Attention
GENERATIVE MODELS
11. Generative Models: Adversarial Networks (GANs), Wasserstein GANs, Variational Auto-Encoders
STATISTICAL PHYSICS OF DEEP LEARNING
12. Statistical Physics of Deep Learning: Mean fields revisited, Energy landscapes, phase transitions in learning
Audience
Advanced Undergraduate
, Graduate
, Postdoc
, Researcher
Video Public
No
Notes Public
Yes
Language
English
Lecturer Intro
Dr Shailesh Lal received his PhD from the Harish-Chandra Research Institute. His research interests are applications of machine learning to string theory and mathematical physics, black holes in string theory and higher-spin holography.