Frontiers in Large Language Models (LLMs)
This course covers cutting-edge developments and research advancements in large language models (LLMs), including popular models, their application technologies, and recent improvements. By completing this course, participants will gain a comprehensive understanding of the latest knowledge in the field of large language models and insights into future development trends.
Lecturer
Date
18th September ~ 16th December, 2024
Location
| Weekday | Time | Venue | Online | ID | Password |
|---|---|---|---|---|---|
| Monday,Wednesday | 13:30 - 15:05 | A3-1a-205 | ZOOM 02 | 518 868 7656 | BIMSA |
Prerequisite
Computer Science, Machine Learning, Natural Language Processing, Python
Syllabus
1. Introduction of Frontier LLMs 1 - GPT Model
2. Introduction of Frontier LLMs 2 - Llama/PaLM/ChatGLM/Kimi
3. Introduction of Frontier LLMs 3 - ViT/Wav2Vec
4. LLM Applications - Prompt Learning 1
5. LLM Applications - Prompt Learning 2
6. LLM Applications - Retrieval-Augmented Generation 1
7. LLM Applications - Retrieval-Augmented Generation 2
8. Advances in LLMs - MoE (Mixture of Experts)
9. Advances in LLMs - Attention as an RNN
10. Advances in LLMs - Infini-attention
11. Advances in LLMs - REFORMER / Wide-Feedforward
12. Advances in LLMs - RoFormer
2. Introduction of Frontier LLMs 2 - Llama/PaLM/ChatGLM/Kimi
3. Introduction of Frontier LLMs 3 - ViT/Wav2Vec
4. LLM Applications - Prompt Learning 1
5. LLM Applications - Prompt Learning 2
6. LLM Applications - Retrieval-Augmented Generation 1
7. LLM Applications - Retrieval-Augmented Generation 2
8. Advances in LLMs - MoE (Mixture of Experts)
9. Advances in LLMs - Attention as an RNN
10. Advances in LLMs - Infini-attention
11. Advances in LLMs - REFORMER / Wide-Feedforward
12. Advances in LLMs - RoFormer
Reference
[1] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin: Attention is All you Need. NIPS 2017: 5998-6008
[2] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL-HLT (1) 2019: 4171-4186
[3] Tom B. Brown, et. al.: Language Models are Few-Shot Learners. NeurIPS 2020
[4] Long Ouyang, et. al.: Training language models to follow instructions with human feedback. NeurIPS 2022
[5] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig: Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Comput. Surv. 55(9): 195:1-195:35 (2023)
[2] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL-HLT (1) 2019: 4171-4186
[3] Tom B. Brown, et. al.: Language Models are Few-Shot Learners. NeurIPS 2020
[4] Long Ouyang, et. al.: Training language models to follow instructions with human feedback. NeurIPS 2022
[5] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig: Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Comput. Surv. 55(9): 195:1-195:35 (2023)
Audience
Advanced Undergraduate
, Graduate
, Postdoc
, Researcher
Video Public
Yes
Notes Public
Yes
Language
Chinese
, English
Lecturer Intro
Dr. Haihua Xie receives a Ph.D. in Computer Science at Iowa State University in 2015. Before joining BIMSA in Oct. 2021, Dr. Xie worked in the State Key Lab of Digital Publishing Technology, Peking University from 2015-2021. His research interests include Natural Language Processing and Knowledge Service. He published more than 20 papers and obtained 7 invention patents. In 2018, Dr. Xie was selected in the 13th batch of overseas high-level talents in Beijing and was hornored as a "Beijing Distinguished Expert".