event thumbnail image
The 25th International Conference on Machine Learning (ICML 2008)

Efficient Bandit Algorithms for Online Multiclass Prediction

author: Shai Shalev-Schwartz, Hebrew University of Jerusalem

Description

This paper introduces the Banditron, a variant of the Perceptron, for the multiclass bandit setting. The multiclass bandit setting models a wide range of practical supervised learning applications where the learner only receives partial feedback (referred to as "bandit" feedback, in the spirit of multi-armed bandit models) with respect to the true label (e.g. in many web applications users often only provide positive "click" feedback which does not necessarily fully disclose a true label). The Banditron has the ability to learn in a multiclass classification setting with the "bandit" feedback which only reveals whether or not the prediction made by the algorithm was correct or not (but does not necessarily reveal the true label). We provide (relative) mistake bounds which show how the Banditron enjoys favorable performance, and our experiments demonstrate the practicality of the algorithm. Furthermore, this paper pays close attention to the important special case when the data is linearly separable --- a problem which has been exhaustively studied in the full information setting yet is novel in the bandit setting.

You might be experiencing some problems with Your Video player.
Slides
0:00 Efficient Bandit Algorithms for Online Multiclass Prediction
0:12 Motivation
1:32 Outline
2:18 Online Bandit Multiclass Categorization
3:31 Linear Hypotheses
4:33 The Multiclass Perceptron
5:50 Perceptron in the Bandit Setting
6:29 Exploration
7:42 Exploration vs. Exploitation
8:51 The Banditron (1)
10:04 The Banditron (2)
11:06 The Banditron Expected Update
11:42 Analysis: The Hinge-Loss (1)
12:18 Analysis: The Hinge-Loss (2)
12:32 Mistake Bounds
13:02 Mistake Bounds (cont.)
14:41 Experiments
15:38 Experimental Results -- Reuters
17:06 Experimental Results -- Separable Data
17:33 Experimental Results -- 5% label noise
18:09 Exploration-Exploitation Parameter
18:42 The Separable Case (1)
19:50 The Separable Case (2)
20:45 The Separable Case (3)
21:56 Extensions and Open Problems

Lecture rating

People found this lecture:
Worth seeing
because it is:
 Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment: