Deep Learning via Semi-Supervised Embedding
published: Aug. 26, 2009, recorded: June 2009, views: 608
Report a problem or upload filesIf you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
We show how nonlinear embedding algorithms popular for use with shallow semi-supervised learning techniques such as kernel methods can be applied to deep multi-layer architectures, either as a regularizer at the output layer, or on each layer of the architecture. This provides a simple alternative to existing approaches to deep learning whilst yielding competitive error rates compared to those methods, and existing shallow semi-supervised techniques.
We then go on to generalize this approach to take advantage of sequential data: for images, and text.
For images, we take advantage of the temporal coherence that naturally exists in unlabeled video recordings. That is, two successive frames are likely to contain the same object or objects. We demonstrate the effectiveness of this method in a semi-supervised setting on some pose invariant object and face recognition tasks.
For text, we describe a unified approach to tagging: a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, and semantic roles. State-of-the-art performance is attained by learning word embeddings using a text specific semi-supervised task called a language model.
Joint work with: Ronan Collobert, Frederic Ratle, Hossein Mobahi, Pavel Kuksa and Koray Kavukcuoglu.
Download slides: icml09_weston_dlss_01.pdf (2.7 MB)
Link this pageWould you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !