BIMSA >
Advances in Artificial Intelligence
A Comprehensive and Explainable Approach to Evaluating LLMs’ Defense Capabilities
A Comprehensive and Explainable Approach to Evaluating LLMs’ Defense Capabilities
演讲者
冯悦
时间
2024年11月28日 14:00 至 16:00
地点
A3-1-301
线上
Zoom 230 432 7880
(BIMSA)
摘要
Given the importance of large language models (LLMs) safety, evaluating their defense capabilities against jailbreak attacks has become a key area of focus. However, current evaluation methods often fail to generalize to complex scenarios and lack transparency, leading to incomplete and inaccurate assessments. To address these limitations, we introduce JAILJUDGE, a comprehensive and explainable benchmark designed to assess LLMs’ defense capabilities. JAILJUDGE covers a wide array of risk scenarios, including synthetic, adversarial, in-the-wild, and multilingual prompts. It also offers detailed explanations to ensure transparent and reliable evaluations.
演讲者介绍
Yue Feng is an assistant professor at the University of Birmingham. She got her Ph.D. from University College London. Her research interests lie in natural language processing and information retrieval. She has published more than 30 papers in top conferences (e.g., ACL, SIGIR, EMNLP, WSDM, etc). She also won the Amazon Alexa Prize TaskBot Challenge and was awarded the Baidu Outstanding Research Intern Star.