event thumbnail image
The 25th International Conference on Machine Learning (ICML 2008)

Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient

author: Tijmen Tieleman, University of Toronto

Description

A new algorithm for training Restricted Boltzmann Machines is introduced. The algorithm, named Persistent Contrastive Divergence, is different from the standard Contrastive Divergence algorithms in that it aims to draw samples from almost exactly the model distribution. It is compared to some standard Contrastive Divergence algorithms on the tasks of modeling handwritten digits and classifying digit images by learning a model of the joint distribution of images and labels. The Persistent Contrastive Divergence algorithm outperforms other Contrastive Divergence algorithms, and is equally fast and simple.

You might be experiencing some problems with Your Video player.
Slides
0:00 Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient
0:35 A Problem with MRFs
2:23 Details of the Problem
4:47 Approximating Algorithms
7:20 CD/PL Problem, in Pictures - 1
7:36 CD/PL Problem, in Pictures - 2
8:02 CD/PL Problem, in Pictures - 3
8:15 CD/PL Problem, in Pictures - 4
9:04 Solution
12:17 More about the Solution
14:00 In Practice…
15:22 Results on Fully Visible MRFs
17:57 Results on RBMs
18:46 Balancing Now Works
19:13 Conclusion
19:35 - Questions

Lecture rating

People found this lecture:
Worth seeing
because it is:
 Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment: