The landscape of the loss surfaces of multilayer networks
published: Aug. 20, 2015, recorded: July 2015, views: 7440
Report a problem or upload filesIf you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Deep learning has enjoyed a resurgence of interest in the last few years for such applications as image and speech recognition, or natural language processing. The vast majority of practical applications of deep learning focus on supervised learning, where the supervised loss function is minimized using stochastic gradient descent. The properties of this highly non-convex loss function, such as its landscape and the behavior of critical points (maxima, minima, and saddle points), as well as the reason why large- and small-size networks achieve radically different practical performance, are however very poorly understood. It was only recently shown that new results in spin-glass theory potentially may provide an explanation for these problems by establishing a connection between the loss function of the neural networks and the Hamiltonian of the spherical spin-glass models. The connection between both models relies on a number of possibly unrealistic assumptions, yet the empirical evidence suggests that the connection may exist in real. The question we pose is whether it is possible to drop some of these assumptions to establish a stronger connection between both models.
Download slides: colt2015_choromanska_multilayer_networks_01.pdf (337.5 KB)
Link this pageWould you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !
Reviews and comments:
The life of each of us is filled with a huge number of stressful situations. Relaxing and resting are sometimes extremely important to keep your peace of mind https://thefixerwhiskey.com/ and gain strength for your next dash. Bulleit Bourbon review will tell you about the main advantages of this whiskey, what products it combines better, what sensations it gives. This is a good drink that will help you quickly relax and forget about everyday problems.
After entering the professional website of the 9Winz on line casino, you discover yourself in the digital international of gambling leisure. The modern https://9winz.net/ organization became founded in 2020, however after some months it gathered a huge army of its admirers. Every day, lots of recent gamers sign up in the famous membership, receiving appealing bonuses and playing the user-friendly interface of the website online. In the 9Winz on-line on line casino in India you cannot only relax, however also win a decent amount of money.
Write your own review or comment: