event thumbnail image
The 25th International Conference on Machine Learning (ICML 2008)

The Projectron: a Bounded Kernel-Based Perceptron

author: Francesco Orabona, IDIAP Research Institute

Description

We present a discriminative online algorithm with a bounded memory growth, which is based on the kernel-based Perceptron. Generally, the required memory of the kernel-based Perceptron for storing the online hypothesis is not bounded. Previous work has been focused on discarding part of the instances in order to keep the memory bounded. In the proposed algorithm the instances are not discarded, but projected onto the space spanned by the previous online hypothesis. We derive a relative mistake bound and compare our algorithm both analytically and empirically to the state-of-the-art Forgetron algorithm (Dekel et al, 2007). The first variant of our algorithm, called Projectron, outperforms the Forgetron. The second variant, called Projectron++, outperforms even the Perceptron.

You might be experiencing some problems with Your Video player.
Slides
0:00 The Projectron: a Bounded Kernel-Based Perceptron
0:08 Outline
0:34 Online Learning with Kernels (1)
1:28 Online Learning with Kernels (2)
1:36 Bounded Online Learning with Kernels
2:07 Discarding Old Samples
2:39 Discard & Bounds
3:14 Discarding is the Only Way?
3:52 Let’s Start from the Linear Perceptron
5:20 Linear Kernel and Projections (1)
6:41 Linear Kernel and Projections (2)
7:02 The Projectron Algorithm
8:12 What is the Effect of η?
8:59 The Support Set Size is Always Bounded!
9:42 Mistake Bound (1)
10:22 Mistake Bound (2)
10:54 Going beyond the Perceptron: the Projectron++
12:13 Comparison
13:33 Size of the Support Set
14:44 Average Online Error
15:10 Size of the Support Set
15:21 Average Online Error
16:02 Size of the Support Set
16:10 Average Online Error
16:11 Some Numerical Results
17:20 Summary
18:16 Thanks for you attention
18:47 - Questions

Lecture rating

People found this lecture:
Worth seeing
because it is:
 Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment: