北京雁栖湖应用数学研究院 北京雁栖湖应用数学研究院

  • 关于我们
    • 院长致辞
    • 理事会
    • 协作机构
    • 参观来访
  • 人员
    • 管理层
    • 科研人员
    • 博士后
    • 来访学者
    • 行政团队
  • 学术研究
    • 研究团队
    • 公开课
    • 讨论班
  • 招生招聘
    • 教研人员
    • 博士后
    • 学生
  • 会议
    • 学术会议
    • 工作坊
    • 论坛
  • 学院生活
    • 住宿
    • 交通
    • 配套设施
    • 周边旅游
  • 新闻
    • 新闻动态
    • 通知公告
    • 资料下载
关于我们
院长致辞
理事会
协作机构
参观来访
人员
管理层
科研人员
博士后
来访学者
行政团队
学术研究
研究团队
公开课
讨论班
招生招聘
教研人员
博士后
学生
会议
学术会议
工作坊
论坛
学院生活
住宿
交通
配套设施
周边旅游
新闻
新闻动态
通知公告
资料下载
清华大学 "求真书院"
清华大学丘成桐数学科学中心
清华三亚国际数学论坛
上海数学与交叉学科研究院
BIMSA > BIMSA Lecture Deep Learning Dynamics: A Scientific Approach
Deep Learning Dynamics: A Scientific Approach
组织者
蔡云峰
演讲者
谢泽柯
时间
2024年06月28日 10:00 至 11:30
地点
A3-4-301
线上
Zoom 293 812 9202 (BIMSA)
摘要
In this talk, I will introduce a series of my works on understanding and improving deep learning via scientific principles and methodology. The success of deep learning depends on both neural networks and optimization dynamcis. I will visit several very foundamental issues in deep learning dynamics: (1) SGD Dynamics and how it selects flat minima; (2) Adam dynamics and how it explains the power of Adam; (3) improving deep learning from a optimization dynamical perspetive; (4) the overlooked pitfalls of weight decay and how to mitigate them; (5) a bridge between protein dynamics and deep learning dynamics. Through this talk, we will also see that scientific principles and theories can provide useful insights and tools for understanding and improving deep learning.
演讲者介绍
I am an Assistant Professor jointly appointed by DSA Thrust and AI Thrust, Hong Kong University of Science and Technology (Guangzhou). I will also be an affiliated Assistant Professor at Dept. of CSE, Hong Kong University of Science and Technology (Clear Water Bay).

Previously, I was a researcher at Baidu Research responsible for large models and AIGC research. I obtained Ph.D. and M.E. both from The University of Tokyo, respectively. I was fortunate to be advised by Prof. Issei Sato and Prof. Masashi Sugiyama. I was also affiliated with RIKEN AIP during the Ph.D. study. Before that, I obtained Bachelor of Science from University of Science and Technology of China.

My mission is to find a way towards the science of AI. I am generally interested in understanding and solving fundamental issues of modern AI by scientific principles and methodology. My current research projects focus on foundations of machine learning and generative AI.
北京雁栖湖应用数学研究院
CONTACT

No. 544, Hefangkou Village Huaibei Town, Huairou District Beijing 101408

北京市怀柔区 河防口村544号
北京雁栖湖应用数学研究院 101408

Tel. 010-60661855
Email. administration@bimsa.cn

版权所有 © 北京雁栖湖应用数学研究院

京ICP备2022029550号-1

京公网安备11011602001060 京公网安备11011602001060