On the regularization of convolutional layers
演讲者
郭培昌
时间
2024年12月23日 15:00 至 16:00
地点
Online
线上
Zoom 928 682 9093
(BIMSA)
摘要
Convolutional neural network is an important model in deep learning, where a convolution operation can be represented by a tensor. To avoid exploding/vanishing gradient problems and to improve the generalizability of a neural network, it is desirable to let the singular values of the transformation matrix corresponding to the tensor be bounded. We propose penalty functions to constrain the singular values of the transformation matrix. We derive the gradient descent algorithm for each penalty function in terms of the tensor. Numerical examples are presented to demonstrate the effectiveness of the method.