BIMSA > Seminar on Control Theory and Nonlinear Filtering The Curse of Memory in Recurrent Neural Networks
The Curse of Memory in Recurrent Neural Networks
Organizer
Shing Toung Yau
Speaker
Zeju Sun
Time
Tuesday, December 27, 2022 9:00 PM - 9:30 PM
Venue
Online
Abstract
In this talk, we will discuss the paper entitled "On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysis" by Z. Li, et al. In this paper, the authors consider the setting of using continuous-time linear RNNs to learn from data generated by linear relationships. The paper uncovered the fact that when there is long-term memory in the target, it takes a large number of neurons to approximate it, and the phenomenon is called the "curse of memory". The analysis serves to be a basic step towards a concrete mathematical understanding of the deep learning using recurrent architectures.