Incorporating Domain Knowledge into Topic Modeling via Dirichlet Forest Priors

author: David Andrzejewski, Computer Sciences Department, University of Wisconsin - Madison

Description

Users of topic modeling methods often have knowledge about the composition of words that should have high or low probability in various topics. We incorporate such domain knowledge using a novel Dirichlet Forest prior in a Latent Dirichlet Allocation framework. The prior is a mixture of Dirichlet tree distributions with special structures. We present its construction, and inference via collapsed Gibbs sampling. Experiments on synthetic and real datasets demonstrate our model’s ability to follow and generalize beyond userspecified domain knowledge.

You might be experiencing some problems with Your Video player.

*Slides*
0:00	Incorporating Domain Knowledge into Topic Modeling via Dirichlet Forest Priors
0:30	New Year’s Wishes (1)
0:52	New Year’s Wishes (2)
0:55	New Year’s Wishes (3)
0:57	New Year’s Wishes (4)
0:59	New Year’s Wishes (5)
1:02	New Year’s Wishes (6)
1:04	New Year’s Wishes (7)
1:07	New Year’s Wishes (8)
1:10	Topic Modeling of Wishes (1)
1:31	Topic Modeling of Wishes (2)
1:51	Topic Modeling of Wishes (3)
2:14	Topic Modeling of Wishes (4)
2:26	Topic Modeling of Wishes (5)
2:41	Topic Modeling with Domain Knowledge (1)
2:44	Topic Modeling with Domain Knowledge (2)
2:48	Topic Modeling with Domain Knowledge (3)
3:01	Topic Modeling with Domain Knowledge (4)
3:38	Topic Modeling with Domain Knowledge (5)
3:59	Topic Modeling with Domain Knowledge (6)
4:11	Word Preferences within Topics (1)
4:15	Word Preferences within Topics (2)
5:53	Word Preferences within Topics (3)
6:37	Word Preferences within Topics (4)
7:05	Word Preferences within Topics (5)
7:16	Word Preferences within Topics (6)
7:58	Dirichlet Prior (“dice factory”)
9:04	Latent Dirichlet Allocation (LDA) (1)
9:24	Latent Dirichlet Allocation (LDA) (2)
9:35	Latent Dirichlet Allocation (LDA) (3)
9:39	Latent Dirichlet Allocation (LDA) (4)
9:44	Latent Dirichlet Allocation (LDA) (5)
9:48	Latent Dirichlet Allocation (LDA) (6)
9:55	Latent Dirichlet Allocation (LDA) (7)
10:12	Latent Dirichlet Allocation (LDA) (8)
10:15	LDA with Dirichlet Forest Prior
10:25	Related work: Correlated Topic Model (CTM)
10:32	Related work: Pachinko Allocation Model (PAM)
10:35	Must-Link (college,school) (1)
10:41	Must-Link (college,school) (2)
10:50	Must-Link (college,school) (3)
11:13	Must-Link (college,school) (4)
11:21	Must-Link (college,school) (5)
11:31	Dirichlet Tree (“dice factory 2.0”) (1)
11:47	Dirichlet Tree (“dice factory 2.0”) (2)
11:52	Dirichlet Tree (“dice factory 2.0”) (3)
12:10	Dirichlet Tree (“dice factory 2.0”) (4)
12:13	Dirichlet Tree (“dice factory 2.0”) (5)
12:54	Dirichlet Tree (“dice factory 2.0”) (6)
13:10	Must-Link (school,college) via Dirichlet Tree (1)
13:42	Must-Link (school,college) via Dirichlet Tree (2)
13:52	Cannot-Link (school,cancer) (1)
14:08	Cannot-Link (school,cancer) (2)
14:14	Cannot-Link (school,cancer) (3)
14:40	Sampling a Tree from the Forest (1)
15:09	Sampling a Tree from the Forest (2)
16:03	Sampling a Tree from the Forest (3)
16:06	Sampling a Tree from the Forest (4)
16:28	Sampling a Tree from the Forest (5)
16:36	Sampling a Tree from the Forest (6)
16:43	Sampling a Tree from the Forest (7)
16:47	Sampling a Tree from the Forest (8)
16:52	Sampling a Tree from the Forest (9)
17:08	Sampling a Tree from the Forest (7)
17:10	Sampling a Tree from the Forest (10)
17:18	Sampling a Tree from the Forest (11)
17:49	LDA with Dirichlet Forest Prior
18:18	Collapsed Gibbs Sampling of (z; q)
18:34	Synthetic Data - Must-Link (B,C)
19:27	Synthetic Data - isolate(B)
19:55	Original Wish Topics (1)
20:05	Original Wish Topics (2)
20:17	isolate([to and for] : : :) (1)
20:28	isolate([to and for] : : :) (2)
20:36	split([cancer free cure well],[go school into college]) (1)
20:43	split([cancer free cure well],[go school into college]) (2)
21:03	merge([love : : : marry: : :],[meet : : : married: : :])
21:31	Conclusions/Acknowledgments

Lecture rating

People found this lecture:
Worth seeing
because it is:
Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.