Gaussian processes in machine learning
Machine learning considers many models. Some are interpretable, others are probabilistic, and others are used in practice. Gaussian process-based models have all these properties: they are interpretable, probabilistic, and lead to practical solutions. The history of applications of Gaussian process regression in machine learning lasts for more than 60 years, while some recent works revolutionized the scope of practical applications of it and theoretical tools to answer fundamental questions. Applications include helping train the models with superhuman powers in the Go game and reducing the cost of aircraft construction by 10%, saving millions of dollars in both cases.
I plan to achieve two main goals in this course: provide an interesting researcher with a powerful tool named Gaussian process-based machine learning models and equip him with a theoretical understanding of the success of Gaussian process regression. In particular, we'll find out how this approach solves the regression problem interpretably, how to estimate the uncertainty in a principled way, and how to select the optimal design of experiments. The lectures are based on my 10 years of experience in this field.
I plan to achieve two main goals in this course: provide an interesting researcher with a powerful tool named Gaussian process-based machine learning models and equip him with a theoretical understanding of the success of Gaussian process regression. In particular, we'll find out how this approach solves the regression problem interpretably, how to estimate the uncertainty in a principled way, and how to select the optimal design of experiments. The lectures are based on my 10 years of experience in this field.
讲师
Alexey Zaytsev
日期
2023年02月21日 至 03月05日
位置
| Weekday | Time | Venue | Online | ID | Password |
|---|---|---|---|---|---|
| 周二,周三,周五 | 15:20 - 16:55 | 1120 | ZOOM 02 | 518 868 7656 | BIMSA |
修课要求
Probability theory, Mathematical statistics, Machine learning
课程大纲
Lecture 1. Gaussian process regression (GPR). Covariance functions. Prediction and uncertainty estimation for GPR
Lecture 2. Large-scale Gaussian process regression. How to deal with large amounts of training data.
Lecture 3. Interpolation error bounds. Minimax interpolation error bound. Optimal variable fidelity design of experiments. Sketch of the proof.
Lecture 4. Surrogate optimization and optimal adaptive design of experiments.
Lecture 5. Upper and lower bounds for regret in optimal adaptive design of experiments. Sketch of the proof.
Lecture 6. Deep Gaussian process regression. How to unite Gaussian process regression and deep learning
Lecture 2. Large-scale Gaussian process regression. How to deal with large amounts of training data.
Lecture 3. Interpolation error bounds. Minimax interpolation error bound. Optimal variable fidelity design of experiments. Sketch of the proof.
Lecture 4. Surrogate optimization and optimal adaptive design of experiments.
Lecture 5. Upper and lower bounds for regret in optimal adaptive design of experiments. Sketch of the proof.
Lecture 6. Deep Gaussian process regression. How to unite Gaussian process regression and deep learning
参考资料
The main books for the initial part of the course are:
1A. C. Rasmussen “Gaussian processes for Machine learning”, 2005.
1B. Garnett, Roman, Bayesian Optimization, 2023, Cambridge University Press, to appear
However, the field significantly changed during the 17 years that passed since then. Below you’ll find relevant articles for each lecture, there will be more considered:
2. Liu, Haitao, et al. "When Gaussian process meets big data: A review of scalable GPs." IEEE transactions on neural networks and learning systems 31.11 (2020): 4405-4423.
3. Zaytsev, Alexey, and Evgeny Burnaev. "Minimax approach to variable fidelity data interpolation." Artificial Intelligence and Statistics. PMLR, 2017.
4. Koziel, Slawomir, and Leifur Leifsson. Surrogate-based modeling and optimization. New York: Springer, 2013.
5. Srinivas, Niranjan, et al. "Gaussian process optimization in the bandit setting: No regret and experimental design." arXiv preprint arXiv:0912.3995 (2009).
6. Wilson, Andrew Gordon, et al. "Deep kernel learning." Artificial intelligence and statistics. PMLR, 2016.
1A. C. Rasmussen “Gaussian processes for Machine learning”, 2005.
1B. Garnett, Roman, Bayesian Optimization, 2023, Cambridge University Press, to appear
However, the field significantly changed during the 17 years that passed since then. Below you’ll find relevant articles for each lecture, there will be more considered:
2. Liu, Haitao, et al. "When Gaussian process meets big data: A review of scalable GPs." IEEE transactions on neural networks and learning systems 31.11 (2020): 4405-4423.
3. Zaytsev, Alexey, and Evgeny Burnaev. "Minimax approach to variable fidelity data interpolation." Artificial Intelligence and Statistics. PMLR, 2017.
4. Koziel, Slawomir, and Leifur Leifsson. Surrogate-based modeling and optimization. New York: Springer, 2013.
5. Srinivas, Niranjan, et al. "Gaussian process optimization in the bandit setting: No regret and experimental design." arXiv preprint arXiv:0912.3995 (2009).
6. Wilson, Andrew Gordon, et al. "Deep kernel learning." Artificial intelligence and statistics. PMLR, 2016.
听众
Graduate
视频公开
公开
笔记公开
公开
语言
英文
讲师介绍
Alexey has deep expertise in machine learning and processing of sequential data. He publishes at top venues, including KDD, ACM Multimedia and AISTATS. Industrial applications of his results are now in service at companies Airbus, Porsche and Saudi Aramco among others.