event thumbnail image
Machine Learning Summer School on Theory and Practice of Computational Learning

Seeking Interpretable Models for High Dimensional Data

author: Bin Yu, Department of Statistics, UC Berkeley, University of California

Description

Extracting useful information from high-dimensional data is the focus of today's statistical research and practice. After broad success of statistical machine learning on prediction through regularization, interpretability is gaining attention and sparsity has been used as its proxy. With the virtues of both regularization and sparsity, Lasso (L1 penalized L2 minimization) has been very popular recently. In this talk, I would like to discuss the theory and pratcice of sparse modeling. First, I will give an overview of recent research on sparsity and explain what useful insights have been learned from theoretical analyses of Lasso. Second, I will present collaborative research with the Gallant Lab at Berkeley on building sparse models (linear, nonlinear, and graphical) that describe fMRI responses in primary visual cortex area V1 to natural images.

You might be experiencing some problems with Your Video player.
Slides
0:00 Seeking Interpretable Models for High Dimensional Data
0:10 Characteristics of Modern Data Problems
0:48 Today’s Talk
2:16 Understanding visual pathway
3:10 Understanding visual pathway through fMRI
4:01 Gallant Lab in Nature News
4:58 Stimuli
5:09 Stimulus to fMRI response
5:43 Gabor Wavelet Pyramid
6:25 Features
6:57 “Neural” (fMRI) encoding for visual cortex V1
7:46 Linear Encoding Model by Gallant Lab
8:50 Modeling “history” at Gallant Lab
10:53 Occam’s Razor
11:40 Occam’s Razor via Model Selection in Linear Regression
12:21 Model Selection Criteria
13:58 Model Selection for image-fMRI problem
16:05 Lasso: L1-norm as a penalty
17:18 Lasso: computation and evaluation
18:55 Model Selection Consistency of Lasso
19:53 Model Selection Consistency of Lasso
20:51 Irrepresentable condition (s=2, p=3): geomery
25:23 Consistency of Lasso for Model Selection
26:58 Model Selection Consistency of Lasso (p>>n)
29:34 Gaussian Graphical Model
30:15 L1 penalized log Gaussian Likelihood
30:59 Success prob’s dependence on n and p (Gaussian)
31:30 Success prob’s dependence on “model complexity” K and n
32:38 Back to image-fMRI problem:Linear sparse encoding model on complex “cells”
33:37 Our story on image-fMRI problem
34:17 Other methods
35:34 Validation correlation
36:08 Comparison of the feature locations
37:25 Our Story (cont)
38:36 SpAM V1 encoding model
40:20 Prediction performance (R2)
40:53 Nonlinearities
41:46 Identical Nonlinearity
42:13 Identical-nonlinearity vs linearity: R^2 prediction
42:44 Episode 3: nonlinearity via power transformations (classical stage)
42:46 Identical-nonlinearity vs linearity: R^2 prediction
43:27 Episode 3: nonlinearity via power transformations (classical stage)
44:28 Make it Gaussian
45:19 Gaussianization of features improves prediction
46:39 Episode 4: localized predictors
47:20 Localized prediction
47:45 After localization: view responses of a single voxel
49:03 Localization does not work for all voxels
49:42 Who is responsible? (on-going)
50:25 Some unlocalized voxels
51:09 Episode 5: sparse graphical models (on-going)
52:17 Episode 6: building neighbor model (on-going)
52:55 Prediction (Neighbor compared to Global)
53:59 Summary
54:45 Summary (cont)
55:49 Future work
56:34 Acknowledgements

Lecture rating

People found this lecture:
Worth seeing
because it is:
 Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: