BIMSA >
控制理论和非线性滤波讨论班
控制理论和非线性滤波讨论班
Beyond the Quadratic Approximation: The Multiscale Structure of Neural Network Loss Landscapes
Beyond the Quadratic Approximation: The Multiscale Structure of Neural Network Loss Landscapes
组织者
丘成栋
演讲者
康家熠
时间
2023年11月03日 21:00 至 21:30
地点
Online
摘要
The quadratic approximation of neural network loss landscapes has been extensively used to study the optimization process of these networks. Though, it usually holds in a very small neighborhood of the minimum, it cannot explain many phenomena observed during the optimization process. Numerically, we observe that neural network loss functions possess a multiscale structure, manifested in two ways: (1) in a neighborhood of minima, the loss mixes a continuum of scales and grows subquadratically, and (2) in a larger region, the loss shows several separate scales clearly.
演讲者介绍
Jiayi Kang received his Ph.D. in Mathematics from Tsinghua University in 2024. He joined the Beijing Institute of Mathematical Sciences and Applications (BIMSA) as an Assistant Researcher in July 2024, and became an Assistant Professor at the Hetao Institute for Mathematical and Interdisciplinary Sciences (HIMIS) in November 2025.
His research focuses on the intersection of deep learning, nonlinear filtering, and computational biology. His main research interests include: neural network-based filtering algorithms and their mathematical foundations, sampling methods in Wasserstein geometry, nonlinear filtering theory (including the Yau-Yau method) and its applications in climate science and other fields, as well as computational genomics and evolutionary system modeling. He is committed to solving complex problems in science and engineering using mathematical and machine learning methods.