	Log in \| Register \|

Carnegie Mellon Machine Learning Lunch seminar

Relational Learning as Collective Matrix Factorization

author: Ajit Singh, The Auton Lab, School of Computer Science, CMU

Description

We present a unified view of matrix factorization models, including singular value decompositions, non-negative matrix factorization, probabilistic latent semantic indexing, and generalizations of these models to exponential families and non-regular Bregman divergences.

One can model relational data as a set of matrices, where each matrix represents the value of a relation between two entity-types. Instead of a single matrix, relational data is represented as a set of matrices with shared dimensions and tied low-rank representation. Our example domain is augmented collaborative filtering, where both user ratings and side information about items are available.

To predict the value of a relation, we extend Bregman matrix factorization to a set of related matrices. Using an alternating minimization scheme, we show the existence of a practical Newton step. The use of stochastic second-order methods for large matrices is also covered.

You might be experiencing some problems with Your Video player.

*Slides*
0:00	Relational Learning using Matrix Factorization
0:26	Relational Data
1:36	Relational Data
2:48	Motivation
4:29	Matrix Factorization (1)
5:39	Matrix Factorization (2)
6:50	Data Weights
7:55	Prediction Link (1)
8:29	Prediction Link (2)
10:24	Weighted Divergence (a.k.a. Loss)
11:10	Optimization Problem
12:51	Regularization
15:50	Hard Constraints
17:02	Example – Weighted Singular Value Decomposition
19:02	Example – Probabilistic Latent Semantic Indexing (1)
23:42	Example – Probabilistic Latent Semantic Indexing (2)
24:10	Example – Probabilistic Latent Semantic Indexing (3)
24:43	Overview - Bregman Divergences, Exponential Families, and Matrix
25:08	Bregman Divergences
26:10	Regular Exponential Families
26:55	Bregman Divergences and Exponential Families
27:52	Duality and Exponential Families
28:37	Bregman Divergences and Exponential Families (1)
29:15	Bregman Divergences and Exponential Families (2)
30:13	Matrix Factorization and Exponential Families
31:23	Overview - Relational Models as Collective Matrix Factorization
31:26	Relational Data and Collective Matrix Factorization
32:08	Collective Matrix Factorization – Optimization (1)
32:23	Collective Matrix Factorization – Optimization (2)
33:00	Alternating Projections for Collective Matrix Factorization
33:07	Collective Matrix Factorization – Optimization (2)
33:37	Alternating Projections for Collective Matrix Factorization
34:04	Projection – Gradient Step
34:07	- Questions
34:25	- Questions
34:31	- Questions
34:45	- Questions
35:01	- Questions
35:14	Alternating Projections for Collective Matrix Factorization
35:29	Projection – Gradient Step
39:46	Projection – Newton Step (1)
42:35	Projection – Newton Step (2)
44:21	Stochastic Optimization (1)
46:21	Stochastic Optimization (2)
47:40	Comparison – Stochastic vs. Newton (1)
48:57	Comparison – Stochastic vs. Newton (2)
50:44	Motivation
53:06	- Questions

Lecture rating

People found this lecture:
Worth seeing
because it is:
Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.