BIMSA >
Seminar on Control Theory and Nonlinear Filtering
The Curse of Memory in Recurrent Neural Networks
The Curse of Memory in Recurrent Neural Networks
Organizer
Stephen S-T. Yau
Speaker
Zeju Sun
Time
Tuesday, December 27, 2022 9:00 PM - 9:30 PM
Venue
Online
Abstract
In this talk, we will discuss the paper entitled "On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysis" by Z. Li, et al. In this paper, the authors consider the setting of using continuous-time linear RNNs to learn from data generated by linear relationships. The paper uncovered the fact that when there is long-term memory in the target, it takes a large number of neurons to approximate it, and the phenomenon is called the "curse of memory". The analysis serves to be a basic step towards a concrete mathematical understanding of the deep learning using recurrent architectures.