Hierarchical Mixture Models: a Probabilistic Analysis

author:Mark Sandler, Google
published: Aug. 15, 2007,   recorded: August 2007,   views: 237
Categories
You might be experiencing some problems with Your Video player.

Related content

Visitors who watched this lecture also watched...
16:56
Mixture Models and Collaborative Filtering Algorithms

324 views - Devdatt Dubhashi, 2006
04:59:19
Machine Learning, Probability and Graphical Models

18450 views - Sam Roweis, 2006
01:05:42
Dirichlet Processes, Chinese Restaurant Processes, and all that

4883 views - Michael I. Jordan, 2005
21:22
Iterative Algorithms for Collaborative Filtering with Mixture Models

157 views - Libertad Tansini, 2006
01:00:47
Gaussian Process Basics

12626 views - David MacKay, 2006
03:24:20
Lectures on Clustering

5739 views - Ulrike von Luxburg, 2007
01:34:49
Dirichlet Processes: Tutorial and Practical Course

6433 views - Yee Whye Teh, 2007
16:51
Interview with Usama Fayyad

1194 views - Davor Orlič, Usama Fayyad, 2007
06:21
Poster: Dirichlet Process Mixture Models for Verb Clustering

107 views - Andreas Vlachos, 2008
05:15:54
Graphical models

4699 views - Zoubin Ghahramani, 2007

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.

Description

Mixture models form one of the most widely used classes of generative models for describing structured and clustered data. In this paper we develop a new approach for the analysis of hierarchical mixture models. More specifically, using a text clustering problem as a motivation, we describe a natural generative process that creates a hierarchical mixture model for the data. In this process, an adversary starts with an arbitrary base distribution and then builds a topic hierarchy via some evolutionary process, where he controls the parameters of the process. We prove that under our assumptions, given a subset of topics that represent generalizations of one another (such as baseball - sports - base), for any document which was produced via some topic in this hierarchy, we can efficiently determine the most specialized topic in this subset, it still belongs to. The quality of the classification is independent of the total number of topics in the hierarchy and our algorithm does not need to know the total number of topics in advance. Our approach also yields an algorithm for clustering and unsupervised topical tree reconstruction. We validate our model by showing that properties predicted by our theoretical results carry over to real data. We then apply our clustering algorithm to two different datasets: (i) “20 newsgroups” [19] and (ii) a snapshot of abstracts of arXiv [2] (15 categories, 240,000 abstracts). In both cases our algorithm performs extremely well.

Link this page  

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: