Frontiers in Large Language Models (LLMs)
This course covers cutting-edge developments and research advancements in large language models (LLMs), including popular models, their application technologies, and recent improvements. By completing this course, participants will gain a comprehensive understanding of the latest knowledge in the field of large language models and insights into future development trends.
讲师
日期
2024年09月18日 至 12月16日
位置
| Weekday | Time | Venue | Online | ID | Password |
|---|---|---|---|---|---|
| 周一,周三 | 13:30 - 15:05 | A3-1a-205 | ZOOM 02 | 518 868 7656 | BIMSA |
修课要求
Computer Science, Machine Learning, Natural Language Processing, Python
课程大纲
1. Introduction of Frontier LLMs 1 - GPT Model
2. Introduction of Frontier LLMs 2 - Llama/PaLM/ChatGLM/Kimi
3. Introduction of Frontier LLMs 3 - ViT/Wav2Vec
4. LLM Applications - Prompt Learning 1
5. LLM Applications - Prompt Learning 2
6. LLM Applications - Retrieval-Augmented Generation 1
7. LLM Applications - Retrieval-Augmented Generation 2
8. Advances in LLMs - MoE (Mixture of Experts)
9. Advances in LLMs - Attention as an RNN
10. Advances in LLMs - Infini-attention
11. Advances in LLMs - REFORMER / Wide-Feedforward
12. Advances in LLMs - RoFormer
2. Introduction of Frontier LLMs 2 - Llama/PaLM/ChatGLM/Kimi
3. Introduction of Frontier LLMs 3 - ViT/Wav2Vec
4. LLM Applications - Prompt Learning 1
5. LLM Applications - Prompt Learning 2
6. LLM Applications - Retrieval-Augmented Generation 1
7. LLM Applications - Retrieval-Augmented Generation 2
8. Advances in LLMs - MoE (Mixture of Experts)
9. Advances in LLMs - Attention as an RNN
10. Advances in LLMs - Infini-attention
11. Advances in LLMs - REFORMER / Wide-Feedforward
12. Advances in LLMs - RoFormer
参考资料
[1] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin: Attention is All you Need. NIPS 2017: 5998-6008
[2] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL-HLT (1) 2019: 4171-4186
[3] Tom B. Brown, et. al.: Language Models are Few-Shot Learners. NeurIPS 2020
[4] Long Ouyang, et. al.: Training language models to follow instructions with human feedback. NeurIPS 2022
[5] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig: Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Comput. Surv. 55(9): 195:1-195:35 (2023)
[2] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL-HLT (1) 2019: 4171-4186
[3] Tom B. Brown, et. al.: Language Models are Few-Shot Learners. NeurIPS 2020
[4] Long Ouyang, et. al.: Training language models to follow instructions with human feedback. NeurIPS 2022
[5] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig: Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Comput. Surv. 55(9): 195:1-195:35 (2023)
听众
Advanced Undergraduate
, Graduate
, 博士后
, Researcher
视频公开
公开
笔记公开
公开
语言
中文
, 英文
讲师介绍
谢海华2015年在美国爱荷华州立大学取得计算机博士学位，之后在北京大学数字出版技术国家重点实验室担任高级研究员和知识服务方向负责人，于2021年10月全职入职BIMSA。他的研究方向包括：自然语言处理和知识服务。他发表论文数量超过20篇，拥有7项发明专利，入选北京市高水平人才项目并当选北京市杰出专家。