Seeking Interpretable Models for High Dimensional Data
Description
Extracting useful information from high-dimensional data is the focus of today's statistical research and practice. After broad success of statistical machine learning on prediction through regularization, interpretability is gaining attention and sparsity has been used as its proxy. With the virtues of both regularization and sparsity, Lasso (L1 penalized L2 minimization) has been very popular recently. In this talk, I would like to discuss the theory and pratcice of sparse modeling. First, I will give an overview of recent research on sparsity and explain what useful insights have been learned from theoretical analyses of Lasso. Second, I will present collaborative research with the Gallant Lab at Berkeley on building sparse models (linear, nonlinear, and graphical) that describe fMRI responses in primary visual cortex area V1 to natural images.
| Slides | |
| 0:00 | Seeking Interpretable Models for High Dimensional Data |
| 0:10 | Characteristics of Modern Data Problems |
| 0:48 | Today’s Talk |
| 2:16 | Understanding visual pathway |
| 3:10 | Understanding visual pathway through fMRI |
| 4:01 | Gallant Lab in Nature News |
| 4:58 | Stimuli |
| 5:09 | Stimulus to fMRI response |
| 5:43 | Gabor Wavelet Pyramid |
| 6:25 | Features |
| 6:57 | “Neural” (fMRI) encoding for visual cortex V1 |
| 7:46 | Linear Encoding Model by Gallant Lab |
| 8:50 | Modeling “history” at Gallant Lab |
| 10:53 | Occam’s Razor |
| 11:40 | Occam’s Razor via Model Selection in Linear Regression |
| 12:21 | Model Selection Criteria |
| 13:58 | Model Selection for image-fMRI problem |
| 16:05 | Lasso: L1-norm as a penalty |
| 17:18 | Lasso: computation and evaluation |
| 18:55 | Model Selection Consistency of Lasso |
| 19:53 | Model Selection Consistency of Lasso |
| 20:51 | Irrepresentable condition (s=2, p=3): geomery |
| 25:23 | Consistency of Lasso for Model Selection |
| 26:58 | Model Selection Consistency of Lasso (p>>n) |
| 29:34 | Gaussian Graphical Model |
| 30:15 | L1 penalized log Gaussian Likelihood |
| 30:59 | Success prob’s dependence on n and p (Gaussian) |
| 31:30 | Success prob’s dependence on “model complexity” K and n |
| 32:38 | Back to image-fMRI problem:Linear sparse encoding model on complex “cells” |
| 33:37 | Our story on image-fMRI problem |
| 34:17 | Other methods |
| 35:34 | Validation correlation |
| 36:08 | Comparison of the feature locations |
| 37:25 | Our Story (cont) |
| 38:36 | SpAM V1 encoding model |
| 40:20 | Prediction performance (R2) |
| 40:53 | Nonlinearities |
| 41:46 | Identical Nonlinearity |
| 42:13 | Identical-nonlinearity vs linearity: R^2 prediction |
| 42:44 | Episode 3: nonlinearity via power transformations (classical stage) |
| 42:46 | Identical-nonlinearity vs linearity: R^2 prediction |
| 43:27 | Episode 3: nonlinearity via power transformations (classical stage) |
| 44:28 | Make it Gaussian |
| 45:19 | Gaussianization of features improves prediction |
| 46:39 | Episode 4: localized predictors |
| 47:20 | Localized prediction |
| 47:45 | After localization: view responses of a single voxel |
| 49:03 | Localization does not work for all voxels |
| 49:42 | Who is responsible? (on-going) |
| 50:25 | Some unlocalized voxels |
| 51:09 | Episode 5: sparse graphical models (on-going) |
| 52:17 | Episode 6: building neighbor model (on-going) |
| 52:55 | Prediction (Neighbor compared to Global) |
| 53:59 | Summary |
| 54:45 | Summary (cont) |
| 55:49 | Future work |
| 56:34 | Acknowledgements |
Lecture rating
| People found this lecture: | ||
| Worth seeing | ||
| because it is: | ||
| Valuable and informative | ||
| Well presented | ||
| Easily understandable | ||
| Acceptably recorded | ||
| You need to login to cast your vote. | ||
Report a problem or upload files
If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Related content
Link this page
Would you like to put a link to this lecture on your homepage?Go ahead! Copy the HTML snippet !





