A Preliminary Evaluation of Word Representations for Named-Entity Recognition

author: Joseph Turian, University of Montreal
published: Jan. 19, 2010, recorded: December 2009, views: 438

Slides

Slides
0:00	A Preliminary Evaluation ofWord Representationsfor Named-Entity Recognition
0:14	Natural language processing - 1
0:35	Natural language processing - 2
0:53	How do we handle words?
1:04	“One-hot”word representation - 1
1:33	“One-hot”word representation - 2
1:55	Approach - 1
2:06	Approach - 2
2:12	Approach - 3
2:40	Less sparse word reprs?
3:13	Class-based word repr - 1
3:35	Class-based word repr - 2
3:45	Less sparse word reprs?
3:52	Brown clustering - 1
4:17	Brown clustering - 2
4:57	Brown clustering - 3
5:16	Less sparse word reprs?
5:19	HMM approach
5:41	Distributed word repr
6:18	Sequence labeling w/ embeddings
6:42	Less sparse word reprs? - 1
6:56	Less sparse word reprs? - 2
6:58	Collobert + Weston 2008 - 1
7:49	Collobert + Weston 2008 - 2
8:21	Less sparse word reprs?
8:26	Log bilinear Language Model (LBL)
10:17	HLBL
10:35	Approach
11:11	Unsupervised corpus
11:21	Supervised Task
11:35	Embeddings: Normalization
12:13	Clean unsupervised corpus?
13:23	Results (late-breaking)
15:09	Hypothesis
16:11	Difficulties with word embeddings
17:07	Summary

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Lecture popularity: You need to login to cast your vote.

Description

We use different word representations as word features for a named-entity recognition (NER) system with a linear model. This work is part of a larger empirical survey, evaluating different word representations on different NLP tasks. We evaluate Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeddings of words. All three representations improve accuracy on NER, with the Brown clusters providing a larger improvement than the two embeddings, and the HLBL embeddings more than the Collobert and Weston (2008) embeddings. We also discuss some of the practical issues in using embeddings as features. Brown clusters are simpler than embeddings because they require less hyperparameter tuning.