Measures of Statistical Dependence

author:Arthur Gretton, Max Planck Institute for Biological Cybernetics, Max Planck Institute
published: Feb. 25, 2007,   recorded: February 2006,   views: 465
Categories
You might be experiencing some problems with Your Video player.

Slides

Slides
0:00 ICA and Kernel Distribution Testing
0:52 Overview
2:10 Some notation and conventions
2:54 ICA
... where to be careful when doing it
3:00 ICA (Population version)
3:52 ICA (empirical version)
4:04 A toy example (1)
4:26 A toy example (2)
4:45 Things that are impossible for ICA (1)
5:17 Things that are impossible for ICA (1) 01
5:37 Things that are impossible for ICA (2)
6:10 Things that are impossible for ICA (3)
6:42 Things that are impossible for ICA (4)
7:15 ICA Step 1 -
Decorrelation
7:20 First step in ICA: decorrelate
7:34 Example: what does decorrelation achieve?
8:03 Decorrelation: a drawback
8:48 What is left: rotation
9:00 Rotation (continued)
9:19 ICA: maximum likelihood
10:48 Maximum likelihood: example
11:21 Maximum likelihood: where it fails
12:21 ICA Step 2(b)
Rotation: contrast functions
12:30 What is a copy?
13:03 Contrast functions
14:08 Contrast functions and maximum likelihood
15:05 Contrast functions and mutual information (1)
15:44 Contrast functions and mutual information (2)
16:52 Contrast functions (3): Some famous cases
17:32 Kurtosis: an important concept
18:20 Contrast functions: Example (1)
18:40 Contrast functions: Example (2)
19:37 Disclaimer!
20:14 ICA for non-i.i.d. signals (1)
22:10 ICA for non-i.i.d. signals (2)
23:21 Advanced (kernel!) -
independence measures
23:25 Kernel dependence measures
24:50 Outline
26:05 Dependence detection
26:35 A second order method
27:20 Take nonlinear features
29:30 The kernel trick (1)
30:12 The kernel trick (2)
30:46 An empirical estimate
31:02 COCO measures independence
31:33 Why universal?
33:10 Background: statistical tests (1)
35:28 Background: statistical tests (2)
37:23 When is dependence hard to detect?
39:48 Hard-to-detect dependence (2)
41:27 Hard-to-detect dependence (3)
42:38 Hard-to-detect dependence (4)
43:12 A test of independence
45:48 Choosing kernel size (1)
47:51 Choosing kernel size (2)
49:02 Application to ICA
50:34 Positive, Negative, and Zero kurtosis
51:46 Outlier resistance
52:25 The Two-Sample Problem
52:31 The two-sample problem
53:41 The MMD (1)
54:31 The MMD (2)
56:05 The MMD (2) 01
57:15 Empirical estimate

Related content

Visitors who watched this lecture also watched...
04:55:29
Kernel Methods for Dependence and Causality

308 views - Kenji Fukumizu, 2007
21:32
A Hilbert-Schmidt Dependence Maximization Approach to Unsupervised Structure Discovery

143 views - Arthur Gretton, 2008
02:14:31
Introduction to kernel methods

2921 views - Alexander J. Smola, 2007
01:00:47
Gaussian Process Basics

12629 views - David MacKay, 2006
05:44:57
Independent Component Analysis

2786 views - Aapo Hyvärinen, 2005
03:32:21
Introduction to Learning Theory

2747 views - Olivier Bousquet, 2006
04:59:19
Machine Learning, Probability and Graphical Models

18451 views - Sam Roweis, 2006
01:35:21
Information Geometry

2218 views - Sanjoy Dasgupta, 2005
05:19:21
Brain Computer Interfaces

836 views - Klaus-Robert Müller, 2006
02:54:53
Some Mathematical Tools for Machine Learning

3712 views - Chris Burges, 2003

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.

 Watch videos:   (click on thumbnail to launch)

Watch Part 1
Part 1 0:57:52 Flash video Slide Synchronization Windows Media video
!NOW PLAYING
Watch Part 2
Part 2 0:04:23 Flash video Slide Synchronization Windows Media video

Description

A number of important problems in signal processing depend on measures of statistical dependence. For instance, this dependence is minimised in the context of instantaneous ICA, in which linearly mixed signals are separated using their (assumed) pairwise independence from each other. A number of methods have been proposed to measure this dependence, however they generally assume a particular parametric model for the densities generating the observations. Recent work suggests that kernel methods may be used to find estimates that adapt according to the signals they compare. These methods are currently being refined, both to yeild greater accuracy, and to permit the use of the signal properties over time in improving signal separability. In addition, these methods can be applied in cases where the statistical dependence between observations must be maximised, which is true for certain classes of clustering algorithms.

Link this page  

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: