Deep Learning Dynamics: A Scientific Approach
组织者
演讲者
谢泽柯
时间
2024年06月28日 10:00 至 11:30
地点
A3-4-301
线上
Zoom 293 812 9202
(BIMSA)
摘要
In this talk, I will introduce a series of my works on understanding and improving deep learning via scientific principles and methodology. The success of deep learning depends on both neural networks and optimization dynamcis. I will visit several very foundamental issues in deep learning dynamics: (1) SGD Dynamics and how it selects flat minima; (2) Adam dynamics and how it explains the power of Adam; (3) improving deep learning from a optimization dynamical perspetive; (4) the overlooked pitfalls of weight decay and how to mitigate them; (5) a bridge between protein dynamics and deep learning dynamics. Through this talk, we will also see that scientific principles and theories can provide useful insights and tools for understanding and improving deep learning.
演讲者介绍
Dr. Zeke Xie is an Assistant Professor at Information Hub, Hong Kong University of Science and Technology (Guangzhou). He is leading Xie Machine Learning Foundations Lab (xLeaF Lab) that generally interested in understanding and solving fundamental issues of modern AI, particularly large models, by scientific principles and methodology. He currently focuses on optimization and inference of Large Models and Generative AI. Previously, he was a researcher at Baidu Research responsible for large models and AIGC research. He obtained Ph.D. and M.E. both from The University of Tokyo. He received multiple faculty research awards from the industry, including ByteDance and Baidu.