A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning

author: Ronan Collobert, NEC Laboratories America, Inc.
published: July 30, 2008,   recorded: July 2008,   views: 13608


Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


We describe a single convolutional neural network architecture that given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning. All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel way of performing semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the generalization of the shared tasks, resulting in a learnt model with state-of-the-art performance.

See Also:

Download slides icon Download slides: icml08_collobert_uanl_01.pdf (899.5┬áKB)

Download subtitles Download subtitles: TT/XML, RT, SRT

Help icon Streaming Video Help

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Reviews and comments:

Comment1 panda handa, November 6, 2014 at 9:36 p.m.:

very nice talk and helpful to understand the paper, but bad quality of sound and visual

Write your own review or comment:

make sure you have javascript enabled or clear this field: