Unreasonable Effectiveness of Learning Artificial Neural Networks
published: Nov. 28, 2016, recorded: November 2016, views: 2657
Report a problem or upload filesIf you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Deep networks are some of the most widely used tools in data science. Learning is in principle a hard problem in these systems, but in practice heuristic algorithms often find solutions with good generalization properties. We propose an explanation of this good performance in terms of a novel large-deviation measure: we show that there are regions of the optimization landscape which are both robust and accessible, and that their existence is crucial to achieve good performance on a class of particularly difficult learning problems. Building on these results, we introduce basic algorithmic schemes which improve existing optimization algorithms and provide a framework for further research on efficient learning for huge data sets and for novel computational technologies.
Download slides: BIDSAconference2016_zecchina_neural_networks_01.pdf (15.5 MB)
Link this pageWould you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !