Language Modeling with Tree Substitution Grammars

author: Matt Post, University of Rochester
published: Jan. 19, 2010, recorded: December 2009, views: 342

Slides

Slides
0:00	Language modeling with tree substitution grammars
0:06	1. certain learned TSGs (a) have lower perplexity and (b) are roughly the same size as the standard Treebank CFG 2
0:31	Standard CFG - 1
0:48	Standard CFG - 2
0:51	Standard CFG - 3
0:58	Standard CFG - 4
1:25	how do we learn a TSG? - 1
1:38	how do we learn a TSG? - 2
2:11	how do we learn a TSG? - 3
2:42	Overview: Post & Gildea (2009) - 1
3:30	Overview: Post & Gildea (2009) - 2
3:49	Overview: Post & Gildea (2009) - 3
4:27	Overview: Post & Gildea (2009) - 4
4:33	Overview: Post & Gildea (2009) - 5
5:05	Overview: Post & Gildea (2009) - 6
5:24	Overview: Post & Gildea (2009) - 7
5:30	Experiments - 1
5:37	Experiments - 2
5:55	Experiments - 3
6:08	Perplexity
7:29	Pseudorandom text
8:12	Perplexity on pseudo-negative text
8:43	Flattening
9:37	2. sampled TSGs lead to perplexity improvements with a bilexical parser, suggesting they are improving Treebank structure
10:07	CFG parsing - 1
10:13	CFG parsing - 2
10:24	Bilexical parsing (Collins Model 1) - 1
10:34	Bilexical parsing (Collins Model 1) - 2
10:38	Bilexical parsing (Collins Model 1) - 3
10:42	Bilexical parsing (Collins Model 1) - 4
10:49	Bilexical parsing (Collins Model 1) - 5
11:11	Bilexical parsing (Collins Model 1) - 6
11:17	Raising
11:56	Conflict - 1
12:05	Conflict - 2
12:23	Conflict - 3
12:30	Conflict - 4
12:39	Conflict - 5
13:12	TSG - 1
13:31	TSG - 2
13:55	Perplexities
14:56	QUESTIONS

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Lecture popularity: You need to login to cast your vote.

Description

We show that a tree substitution grammar (TSG) induced with a collapsed Gibbs sampler results in lower perplexity on test data than both a standard context-free grammar and other heuristically trained TSGs, suggesting that it is better suited to language modeling. Training a more complicated bilexical parsing model across TSG derivations shows further (though nuanced) improvement. We conduct analysis and point to future areas of research using TSGs as language models.