The 25th International Conference on Machine Learning (ICML 2008)

Bi-Level Path Following for Cross Validated Solution of Kernel Quantile Regression

author: Saharon Rosset, Tel Aviv University

Description

Modeling of conditional quantiles requires specification of the quantile being estimated and can thus be viewed as a parameterized predictive modeling problem. Quantile loss is typically used, and it is indeed parameterized by a quantile parameter. In this paper we show how to follow the path of cross validated solutions to regularized kernel quantile regression. Even though the bi-level optimization problem we encounter for every quantile is non-convex, the manner in which the optimal cross-validated solution evolves with the parameter of the loss function allows tracking of this solution. We prove this property, construct the resulting algorithm, and demonstrate it on data. This algorithm allows us to efficiently solve the whole family of bi-level problems.

You might be experiencing some problems with Your Video player.

*Slides*
0:00	Bi-Level Path Following for Cross Validated Solution of Kernel Quantile Regression
0:22	Predictive Modeling - 1
0:35	Predictive Modeling - 2
0:38	Predictive Modeling - 3
0:39	Predictive Modeling - 4
0:45	Predictive Modeling - 5
0:52	Predictive Modeling - 6
1:17	Regularized Optimization - 1
1:29	Regularized Optimization - 2
1:33	Regularized Optimization - 3
1:39	Regularized Optimization - 4
1:43	Regularized Optimization - 5
1:59	Example: Kernel Machines - 1
2:01	Example: Kernel Machines - 2
2:13	Example: Kernel Machines - 3
2:17	Example: Kernel Machines - 4
2:28	Example: Kernel Machines - 5
2:32	Example: Kernel Machines - 6
2:46	Model Selection - 1
3:06	Model Selection - 2
3:10	Model Selection - 3
3:11	Model Selection - 4
3:24	Model Selection - 5
3:33	Regularization Path - 1
4:01	Regularization Path - 2
4:16	Regularization Path - 3
4:33	Regularization Path - 4
4:55	Regularization Path Schematic - 1
5:04	Regularization Path Schematic - 2
5:25	Regularization Path Schematic - 3
5:28	Regularization Path Schematic - 4
5:38	Cross Validation - 1
6:21	Cross Validation - 2
6:29	Cross Validation - 3
7:01	Cross Validation - 4
7:14	Cross Validation - 5
7:16	Conditional Quantile Modeling - 1
7:58	Conditional Quantile Modeling - 2
8:26	Conditional Quantile Modeling - 3
8:32	Conditional Quantile Modeling - 4
8:38	Conditional Quantile Modeling - 5
9:08	Non-Uniform Noise -> Non Parallel Quantiles
9:39	Quantile Regression
10:32	Quantile Regression and Kernel Quantile Regression - 1
10:58	Quantile Regression and Kernel Quantile Regression - 2
11:11	Quantile Regression and Kernel Quantile Regression - 3
11:25	Bi-Level Programming - 1
11:44	Bi-Level Programming - 2
12:31	Bi-Level Programming - 3
13:25	Regularization Paths as τ and λ Vary - 1
13:41	Regularization Paths as τ and λ Vary - 2
14:14	Regularization Paths as τ and λ Vary - 3
14:28	Regularization Paths as τ and λ Vary - 4
15:03	Mapping f (τ,λ) - 1
15:05	Mapping f (τ,λ) - 2
15:10	Mapping f (τ,λ) - 3
15:19	Regularization Paths as τ and λ Vary - 4
15:21	Mapping f (τ,λ) - 1
15:22	Mapping f (τ,λ) - 2
15:22	Mapping f (τ,λ) - 3
15:24	Mapping f (τ,λ) - 4
15:25	Mapping f (τ,λ) - 5
15:26	Mapping f (τ,λ) - 6
15:27	Mapping f (τ,λ) - 7
15:28	Mapping f (τ,λ) - 8
15:38	Mapping f (τ,λ) - 9
15:53	Knot Evolution: Some Details
16:14	Knot Crossing Example
16:41	Properties of the Bi-Level Solutions - 1
16:41	Properties of the Bi-Level Solutions - 2
16:42	Properties of the Bi-Level Solutions - 3
16:43	Properties of the Bi-Level Solutions - 4
16:44	Properties of the Bi-Level Solutions - 5
16:51	Properties of the Bi-Level Solutions - 6
17:18	Properties of the Bi-Level Solutions - 7
17:43	Properties of the Bi-Level Solutions - 8
17:47	Evolution of Bi-Level Solution - 1
17:48	Evolution of Bi-Level Solution - 2
17:49	Evolution of Bi-Level Solution - 3
17:58	Schematic of Complete Algorithm - 1
17:59	Schematic of Complete Algorithm - 2
17:59	Schematic of Complete Algorithm - 3
18:00	Schematic of Complete Algorithm - 4
18:01	Schematic of Complete Algorithm - 5
18:02	Schematic of Complete Algorithm - 6
18:03	Schematic of Complete Algorithm - 7
18:03	Schematic of Complete Algorithm - 8
18:19	Schematic of Complete Algorithm - 9
18:41	Schematic of Complete Algorithm - 10
19:01	Example 1: Parallel Quantiles - 1
19:12	Example 1: Parallel Quantiles - 2
19:57	Example 2: Non-Parallel Quantiles - 1
20:10	icExample 2: Non-Parallel Quantiles - 2
20:30	Summary - 1
20:45	- Questions

Lecture rating

People found this lecture:
Worth seeing
because it is:
Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.