Relational Learning as Collective Matrix Factorization
Description
We present a unified view of matrix factorization models, including singular value decompositions, non-negative matrix factorization, probabilistic latent semantic indexing, and generalizations of these models to exponential families and non-regular Bregman divergences. One can model relational data as a set of matrices, where each matrix represents the value of a relation between two entity-types. Instead of a single matrix, relational data is represented as a set of matrices with shared dimensions and tied low-rank representation. Our example domain is augmented collaborative filtering, where both user ratings and side information about items are available. To predict the value of a relation, we extend Bregman matrix factorization to a set of related matrices. Using an alternating minimization scheme, we show the existence of a practical Newton step. The use of stochastic second-order methods for large matrices is also covered.
| Slides | |
| 0:00 | Relational Learning using Matrix Factorization |
| 0:26 | Relational Data |
| 1:36 | Relational Data |
| 2:48 | Motivation |
| 4:29 | Matrix Factorization (1) |
| 5:39 | Matrix Factorization (2) |
| 6:50 | Data Weights |
| 7:55 | Prediction Link (1) |
| 8:29 | Prediction Link (2) |
| 10:24 | Weighted Divergence (a.k.a. Loss) |
| 11:10 | Optimization Problem |
| 12:51 | Regularization |
| 15:50 | Hard Constraints |
| 17:02 | Example – Weighted Singular Value Decomposition |
| 19:02 | Example – Probabilistic Latent Semantic Indexing (1) |
| 23:42 | Example – Probabilistic Latent Semantic Indexing (2) |
| 24:10 | Example – Probabilistic Latent Semantic Indexing (3) |
| 24:43 | Overview - Bregman Divergences, Exponential Families, and Matrix |
| 25:08 | Bregman Divergences |
| 26:10 | Regular Exponential Families |
| 26:55 | Bregman Divergences and Exponential Families |
| 27:52 | Duality and Exponential Families |
| 28:37 | Bregman Divergences and Exponential Families (1) |
| 29:15 | Bregman Divergences and Exponential Families (2) |
| 30:13 | Matrix Factorization and Exponential Families |
| 31:23 | Overview - Relational Models as Collective Matrix Factorization |
| 31:26 | Relational Data and Collective Matrix Factorization |
| 32:08 | Collective Matrix Factorization – Optimization (1) |
| 32:23 | Collective Matrix Factorization – Optimization (2) |
| 33:00 | Alternating Projections for Collective Matrix Factorization |
| 33:07 | Collective Matrix Factorization – Optimization (2) |
| 33:37 | Alternating Projections for Collective Matrix Factorization |
| 34:04 | Projection – Gradient Step |
| 34:07 | - Questions |
| 34:25 | - Questions |
| 34:31 | - Questions |
| 34:45 | - Questions |
| 35:01 | - Questions |
| 35:14 | Alternating Projections for Collective Matrix Factorization |
| 35:29 | Projection – Gradient Step |
| 39:46 | Projection – Newton Step (1) |
| 42:35 | Projection – Newton Step (2) |
| 44:21 | Stochastic Optimization (1) |
| 46:21 | Stochastic Optimization (2) |
| 47:40 | Comparison – Stochastic vs. Newton (1) |
| 48:57 | Comparison – Stochastic vs. Newton (2) |
| 50:44 | Motivation |
| 53:06 | - Questions |
Lecture rating
| People found this lecture: | ||
| Worth seeing | ||
| because it is: | ||
| Valuable and informative | ||
| Well presented | ||
| Easily understandable | ||
| Acceptably recorded | ||
| You need to login to cast your vote. | ||
Report a problem or upload files
If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
SEE ALSO:
Link this page
Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !





