Machine Learning Summer School 2006 - Canberra

Measures of Statistical Dependence

author: Arthur Gretton, Max Planck Institute for Biological Cybernetics

Description

A number of important problems in signal processing depend on measures of statistical dependence. For instance, this dependence is minimised in the context of instantaneous ICA, in which linearly mixed signals are separated using their (assumed) pairwise independence from each other. A number of methods have been proposed to measure this dependence, however they generally assume a particular parametric model for the densities generating the observations. Recent work suggests that kernel methods may be used to find estimates that adapt according to the signals they compare. These methods are currently being refined, both to yeild greater accuracy, and to permit the use of the signal properties over time in improving signal separability. In addition, these methods can be applied in cases where the statistical dependence between observations must be maximised, which is true for certain classes of clustering algorithms.

Categories

Top: Computer Science: Machine Learning: Kernel Methods

You might be experiencing some problems with Your Video player.

*Slides*
0:00	ICA and Kernel Distribution Testing
0:52	Overview
2:10	Some notation and conventions
2:54	ICA ... where to be careful when doing it
3:00	ICA (Population version)
3:52	ICA (empirical version)
4:04	A toy example (1)
4:26	A toy example (2)
4:45	Things that are impossible for ICA (1)
5:17	Things that are impossible for ICA (1) 01
5:37	Things that are impossible for ICA (2)
6:10	Things that are impossible for ICA (3)
6:42	Things that are impossible for ICA (4)
7:15	ICA Step 1 - Decorrelation
7:20	First step in ICA: decorrelate
7:34	Example: what does decorrelation achieve?
8:03	Decorrelation: a drawback
8:48	What is left: rotation
9:00	Rotation (continued)
9:19	ICA: maximum likelihood
10:48	Maximum likelihood: example
11:21	Maximum likelihood: where it fails
12:21	ICA Step 2(b) Rotation: contrast functions
12:30	What is a copy?
13:03	Contrast functions
14:08	Contrast functions and maximum likelihood
15:05	Contrast functions and mutual information (1)
15:44	Contrast functions and mutual information (2)
16:52	Contrast functions (3): Some famous cases
17:32	Kurtosis: an important concept
18:20	Contrast functions: Example (1)
18:40	Contrast functions: Example (2)
19:37	Disclaimer!
20:14	ICA for non-i.i.d. signals (1)
22:10	ICA for non-i.i.d. signals (2)
23:21	Advanced (kernel!) - independence measures
23:25	Kernel dependence measures
24:50	Outline
26:05	Dependence detection
26:35	A second order method
27:20	Take nonlinear features
29:30	The kernel trick (1)
30:12	The kernel trick (2)
30:46	An empirical estimate
31:02	COCO measures independence
31:33	Why universal?
33:10	Background: statistical tests (1)
35:28	Background: statistical tests (2)
37:23	When is dependence hard to detect?
39:48	Hard-to-detect dependence (2)
41:27	Hard-to-detect dependence (3)
42:38	Hard-to-detect dependence (4)
43:12	A test of independence
45:48	Choosing kernel size (1)
47:51	Choosing kernel size (2)
49:02	Application to ICA
50:34	Positive, Negative, and Zero kurtosis
51:46	Outlier resistance
52:25	The Two-Sample Problem
52:31	The two-sample problem
53:41	The MMD (1)
54:31	The MMD (2)
56:05	The MMD (2) 01
57:15	Empirical estimate

Lecture rating

People found this lecture:
Worth seeing
because it is:
Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.