0.25
0.5
0.75
1.25
1.5
1.75
2
Optimization in Learning and Data Analysis
Published on Sep 27, 201313678 Views
Optimization tools are vital to data analysis and learning. The optimization perspective has provided valuable insights, and optimization formulations have led to practical algorithms with good theore
Related categories
Chapter list
Optimization in Learning and Data Analysis00:00
Big Picture00:05
Outline01:21
Canonical Formulations02:34
Linear Regression03:02
Feature Selection: LASSO04:00
Compressed Sensing04:41
Support Vector Classi06:36
(Regularized) Logistic Regression08:23
Matrix Completion09:35
Inverse Covariance Estimation10:52
Deep Belief Networks - 112:10
Deep Belief Networks - 213:22
Image Processing14:22
Total Variation Regularization15:06
Data Assimilation16:31
Optimization Formulations: Typical Properties18:23
Optimization Toolbox20:43
Gradient Methods: Steepest Descent21:33
Momentum!24:17
Accelerated Gradient Methods26:14
Stochastic Gradient27:02
Classical SG28:20
SG Variants30:01
Coordinate Descent31:01
Coordinate Descent Illustrated31:59
Coordinate Descent: Extensions and Convergence32:17
Regularization33:15
ℓ1 and Sparsity34:34
Other Structures35:06
Shrink Operators35:39
Using Shrinks37:15
Newton's Method37:45
Higher-Order Methods38:39
Augmented Lagrangian40:02
ADMM - 141:16
ADMM - 242:47
ADMM for Awkward Intersections43:29
Matching Tools to Applications - 144:47
Matching Tools to Applications - 245:47
Matching Tools to Applications - 346:10
Multicore Asynchronous Methods47:02
Parallel Stochastic Gradient48:56
Asynchronous Stochastic Gradient: Hogwild!49:50
Hogwild! Convergence51:40
Hogwild! Performance - 152:59
Hogwild! Performance - 253:28
Asynchronous Stochastic Coordinate Descent (ASCD)53:34
Constants and "Diagonalicity"54:29
Diagonalicity Illustrated55:17
How to choose γ?55:45
ASCD Discussion57:00
Implemented on 4-socket, 40-core Intel Xeon57:19
Extreme Linear Programming - 157:52
Extreme Linear Programming - 258:25
LP Rounding Approximations58:57
Sample Results01:00:07
Computation Times (Seconds)01:00:24
Conclusions01:00:44