A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning
Description
We describe a single convolutional neural network architecture that given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning. All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel way of performing semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the generalization of the shared tasks, resulting in a learnt model with state-of-the-art performance.
| Slides | |
| 0:00 | A Unified Architecture for Natural Language Processing |
| 0:33 | The Big Picture - 1 |
| 2:11 | The Big Picture - 2 |
| 2:24 | NLP Tasks |
| 3:52 | The Shallow System Way - 1 |
| 4:26 | The Shallow System Way - 2 |
| 5:36 | The Deep Learning Way - 1 |
| 6:46 | The Deep Learning Way - 2 |
| 7:24 | Convolutions |
| 7:57 | The Deep Learning Way - 2 |
| 8:24 | Convolutions |
| 8:31 | Removing The Time Dimension - 1 |
| 9:16 | Removing The Time Dimension - 2 |
| 9:40 | Multi-Task Learning |
| 10:10 | Improving Word Embedding |
| 11:52 | Language Model: Think Massive |
| 13:17 | Language Model: Embedding |
| 13:57 | MTL: Semantic Role Labeling |
| 15:18 | MTL: Unified Network for NLP |
| 16:04 | - Questions |
Lecture rating
| People found this lecture: | ||
| Worth seeing | ||
| because it is: | ||
| Valuable and informative | ||
| Well presented | ||
| Easily understandable | ||
| Acceptably recorded | ||
| You need to login to cast your vote. | ||
Report a problem or upload files
If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
SEE ALSO:
Link this page
Would you like to put a link to this lecture on your homepage?Go ahead! Copy the HTML snippet !





