Machine Learning Summer School on Theory and Practice of Computational Learning

Theory and Applications of Boosting

author: Robert Schapire, Department of Computer Science, Princeton University

Description

Boosting is a general method for producing a very accurate classification rule by combining rough and moderately inaccurate "rules of thumb". While rooted in a theoretical framework of machine learning, boosting has been found to perform quite well empirically. This tutorial will introduce the boosting algorithm AdaBoost, and explain the underlying theory of boosting, including explanations that have been given as to why boosting often does not suffer from overfitting, as well as some of the myriad other theoretical points of view that have been taken on this algorithm. Some practical applications and extensions of boosting will also be described.

You might be experiencing some problems with Your Video player.

*Slides*
0:00	Theory and Applications of Boosting
0:31	Example: ““How May I Help You?” (1)
2:03	Example: ““How May I Help You?”” (2)
3:03	The Boosting Approach
4:03	Details
5:12	Boosting
6:40	Outline of Tutorial
7:33	Brief Background
7:38	Strong and Weak Learnability (1)
8:40	Strong and Weak Learnability (2)
9:26	Early Boosting Algorithms
10:00	AdaBoost, [Freund & Schapire ’95)
10:26	Basic lgorithm and Core Theory
10:49	A Formal Description of Boosting (1)
11:54	A Formal Description of Boosting (2)
14:10	A Formal Description of Boosting (3)
14:34	AdaBoost [with Freund] (1)
15:06	AdaBoost [with Freund] (2)
17:08	AdaBoost [with Freund] (3)
17:12	Toy Example
17:51	Round 1
18:44	Round 2
19:46	Round 3
20:09	AdaBoost [with Freund] (3)
21:02	Final Classifier
22:10	Analyzing the Training Error (1)
22:55	Analyzing the Training Error (2)
23:35	Analyzing the Training Error (3)
24:59	Proof
25:48	AdaBoost [with Freund] (3)
26:04	Proof
26:27	Proof (cont.) (1)
26:48	Proof (cont.) (2)
27:07	Proof (cont.) (3)
27:26	Proof (cont.) (4)
27:54	Proof (cont.) (5)
28:11	Proof (cont.) (6)
28:18	Proof (cont.) (7)
28:21	Proof (cont.) (6)
28:26	Proof (cont.) (7)
28:35	Proof (cont.) (8)
29:46	How Will Test Error Behave? (A First Guess)
30:07	Proof (cont.) (8)
30:24	How Will Test Error Behave? (A First Guess)
32:52	Actual Typical Run
39:33	A Better Story: The Margins Explanation [with Freund, Bartlett & Lee] (1)
40:38	A Better Story: The Margins Explanation [with Freund, Bartlett & Lee] (2)
41:38	A Better Story: The Margins Explanation [with Freund, Bartlett & Lee] (3)
43:01	Empirical Evidence: The Margin Distribution
45:09	Theoretical Evidence: Analyzing Boosting Using Margins (1)
45:45	Theoretical Evidence: Analyzing Boosting Using Margins (2)
46:44	Theoretical Evidence: Analyzing Boosting Using Margins (3)
46:50	Empirical Evidence: The Margin Distribution
46:56	Theoretical Evidence: Analyzing Boosting Using Margins (3)
47:17	Theoretical Evidence: Analyzing Boosting Using Margins (4)
48:06	Theoretical Evidence: Analyzing Boosting Using Margins (5)
48:07	More Technically...
48:37	Theoretical Evidence: Analyzing Boosting Using Margins (5)
49:07	Other Ways of Understanding AdaBoost

Lecture rating

People found this lecture:
Worth seeing
because it is:
Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.