Structured Prediction for Natural Language Processing

author: Noah Smith, Language Technologies Institute, Carnegie Mellon University

Description

This tutorial will discuss the use of structured prediction methods from machine learning in natural language processing. The field of NLP has, in the past two decades, come to simultaneously rely on and challenge the field of machine learning. Statistical methods now dominate NLP, and have moved the field forward substantially, opening up new possibilities for the exploitation of data in developing NLP components and applications. However, formulations of NLP problems are often simplified for computational or practical convenience, at the expense of system performance. This tutorial aims to introduce several structured prediction problems from NLP, current solutions, and challenges that lie ahead. Applications in NLP are a mainstay at ICML conferences; many ML researchers view NLP as a primary or secondary application area of interest. This tutorial will help the broader ML community understand this important application area, how progress is measured, and the trade-offs that make it a challenge.

You might be experiencing some problems with Your Video player.

*Slides*
0:00	Structured Prediction for Natural Language Processing
0:38	A Very Long Relationship
1:34	Grievances from NLP
2:34	Grievances from ML
3:34	This marriage can survive...
4:18	Where We're Going
5:26	Where We're Not Going
6:06	Managing Expectations
6:34	What's Structured Prediction?
7:22	Representations & Data
7:34	Don't Think About A Bag of Words (1)
8:14	Don't Think About A Bag of Words (2)
8:22	Don't Think About A Bag of Words (3)
8:38	Some Notation
8:58	Where are the Words? (1)
9:50	Where are the Words? (2)
10:34	The Problem with Words (1)
10:50	The Problem with Words (2)
11:18	The Problem with Words (3)
11:22	The Problem with Words (4)
11:42	What's in a Word? (2)
11:44	The Problem with Words (5)
12:06	What's in a Word? (1)
12:58	What's in a Word? (2)
13:10	Extreme Tokenization: Parts of Speech
13:54	Why Structure's Required
14:50	Learning from Data
16:10	Morphology: Dirty Words (1)
17:02	Morphology: Dirty Words (2)
17:30	Agglutinative Morphology
18:18	Interesting Substrings: Chunks (1)
19:02	Interesting Substrings: Chunks (2)
19:11	Interesting Substrings: Chunks (1)
19:22	Interesting Substrings: Chunks (2)
19:34	Named Entity Recognition (1)
19:42	Named Entity Recognition (2)
20:06	Extreme Chunks: Parsing (1)
21:06	Extreme Chunks: Parsing (2)
21:17	NL vs. PL
22:14	Little hope given brain-damaged woman... (1)
22:26	Little hope given brain-damaged woman... (2)
22:46	Little hope given brain-damaged woman... (4)
22:58	Little hope given brain-damaged woman... (5)
23:18	Alternative to Phrases: Dependency Parsing (1)
23:50	Alternative to Phrases: Dependency Parsing (1)
24:46	Alternative to Phrases: Dependency Parsing (2)
24:58	Two Versions of Dependency Parsing (1)
25:18	Alternative to Phrases: Dependency Parsing (1)
25:30	Two Versions of Dependency Parsing (1)
26:06	Two Versions of Dependency Parsing (2)
26:22	Interlude: Linguistic Pipeline (1)
26:42	Interlude: Linguistic Pipeline (2)
28:10	Meaning (1)
29:02	Meaning (2)
30:10	Grounding (1)
30:26	Grounding (2)
31:10	Grounding (3)
31:42	Another Dimension: Multiple Languages
32:58	Debates in NLP
34:14	NLP Problems on the Frontier of Structured Prediction
37:02	Decoding
37:10	Notation
37:42	Why "Decoder"?
40:10	Decoding Defined
41:22	Linear Models
42:22	Simplest Recipe For Structured Prediction
42:46	On "Local" Features
44:30	Dynamic Programming
45:10	(Classical) Viterbi Algorithm (1)
46:06	Viterbi, Visualized
46:30	(Classical) Viterbi Algorithm (2)
47:06	(Generalized) Viterbi Algorithm
47:22	What Features Are "Local"?
48:42	Other DP Algorithms
49:06	Generic Dynamic Programming
49:42	Structures are Graphs
49:54	Maximum Weighted Bipartite Matching (1)
50:18	Maximum Weighted Bipartite Matching (2)
50:50	Maximum Weighted (Directed) Spanning Tree
51:54	Dependency Parsing Features
52:38	Other Approaches
53:50	Current Hot Topics

Lecture rating

People found this lecture:
Worth seeing
because it is:
Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.