The Projectron: a Bounded Kernel-Based Perceptron
Description
We present a discriminative online algorithm with a bounded memory growth, which is based on the kernel-based Perceptron. Generally, the required memory of the kernel-based Perceptron for storing the online hypothesis is not bounded. Previous work has been focused on discarding part of the instances in order to keep the memory bounded. In the proposed algorithm the instances are not discarded, but projected onto the space spanned by the previous online hypothesis. We derive a relative mistake bound and compare our algorithm both analytically and empirically to the state-of-the-art Forgetron algorithm (Dekel et al, 2007). The first variant of our algorithm, called Projectron, outperforms the Forgetron. The second variant, called Projectron++, outperforms even the Perceptron.
| Slides | |
| 0:00 | The Projectron: a Bounded Kernel-Based Perceptron |
| 0:08 | Outline |
| 0:34 | Online Learning with Kernels (1) |
| 1:28 | Online Learning with Kernels (2) |
| 1:36 | Bounded Online Learning with Kernels |
| 2:07 | Discarding Old Samples |
| 2:39 | Discard & Bounds |
| 3:14 | Discarding is the Only Way? |
| 3:52 | Let’s Start from the Linear Perceptron |
| 5:20 | Linear Kernel and Projections (1) |
| 6:41 | Linear Kernel and Projections (2) |
| 7:02 | The Projectron Algorithm |
| 8:12 | What is the Effect of η? |
| 8:59 | The Support Set Size is Always Bounded! |
| 9:42 | Mistake Bound (1) |
| 10:22 | Mistake Bound (2) |
| 10:54 | Going beyond the Perceptron: the Projectron++ |
| 12:13 | Comparison |
| 13:33 | Size of the Support Set |
| 14:44 | Average Online Error |
| 15:10 | Size of the Support Set |
| 15:21 | Average Online Error |
| 16:02 | Size of the Support Set |
| 16:10 | Average Online Error |
| 16:11 | Some Numerical Results |
| 17:20 | Summary |
| 18:16 | Thanks for you attention |
| 18:47 | - Questions |
Lecture rating
| People found this lecture: | ||
| Worth seeing | ||
| because it is: | ||
| Valuable and informative | ||
| Well presented | ||
| Easily understandable | ||
| Acceptably recorded | ||
| You need to login to cast your vote. | ||
Report a problem or upload files
If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Related content
SEE ALSO:
Link this page
Would you like to put a link to this lecture on your homepage?Go ahead! Copy the HTML snippet !





