Statistical Seminar: Every Monday at 2:00 pm.
Time: 2:00 pm – 3:00 pm
Date: 24th March
Place: 3001

Raphaël BERTHIER (INRIA, Sorbonne Université) – Diagonal linear networks and the lasso regularization path

 Abstract: 

Diagonal linear networks are neural networks with linear activation and diagonal weight matrices. The interest in this extremely simple neural network structure is theoretical: its implicit regularization can be rigorously analyzed. In this talk, I will show that the training trajectory of diagonal linear networks is closely related to the lasso regularization path, even when no explicit sparse penalization is used. In this connection, the training time plays the role of an inverse regularization parameter. As a consequence, an earlier stopping time leads to a sparser linear model.

Organizers:

Anna KORBA (CREST), Karim LOUNICI (CMAP) , Jaouad MOURTADA (CREST)

Sponsors:
CREST-CMAP