Technical Program

Paper Detail

Paper Title The Information Bottleneck Theory of Deep Learning: Towards Interpretable Deep Neural Networks
Paper IdentifierTU3.R1.1
Authors Naftali Tishby, Hebrew University of Jerusalem, Israel
Session Applications to Scientific Discovery
Location Le Théatre (Parterre), Level -1
Session Time Tuesday, 09 July, 14:30 - 16:10
Presentation Time Tuesday, 09 July, 14:30 - 14:50
Abstract The Information Bottleneck Theory of Deep Neural Networks suggests that the different layers of the network form a successively refundable code of the information bottleneck accuracy complexity trade-off, analogous to successively refillable codes in RDT. In this talk I will present a new proof that Stochastic Gradient Descent is indeed pushing the layers towards different locations on the Information Curve. I will further argue that this can provide explicit expression for the features encoded by each layer of the network, and explain the convergence time boost due to the hidden layers.