event thumbnail image
The 25th International Conference on Machine Learning (ICML 2008)

A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning

author: Ronan Collobert, NEC Laboratories America, Inc.

Description

We describe a single convolutional neural network architecture that given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning. All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel way of performing semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the generalization of the shared tasks, resulting in a learnt model with state-of-the-art performance.

You might be experiencing some problems with Your Video player.
Slides
0:00 A Unified Architecture for Natural Language Processing
0:33 The Big Picture - 1
2:11 The Big Picture - 2
2:24 NLP Tasks
3:52 The Shallow System Way - 1
4:26 The Shallow System Way - 2
5:36 The Deep Learning Way - 1
6:46 The Deep Learning Way - 2
7:24 Convolutions
7:57 The Deep Learning Way - 2
8:24 Convolutions
8:31 Removing The Time Dimension - 1
9:16 Removing The Time Dimension - 2
9:40 Multi-Task Learning
10:10 Improving Word Embedding
11:52 Language Model: Think Massive
13:17 Language Model: Embedding
13:57 MTL: Semantic Role Labeling
15:18 MTL: Unified Network for NLP
16:04 - Questions

Lecture rating

People found this lecture:
Worth seeing
because it is:
 Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: