Overview of New Developments in Boosting
published: Feb. 21, 2008, recorded: February 2008, views: 758
Report a problem or upload filesIf you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
I will give an overview of recent developments in boosting, focusing on three papers which take very different approaches towards making boosting more efficient and effective. Boosters iteratively choose base classifiers via a weak learner and then update a distribution over training examples. Roughly, the three papers show progress on the three issues implicit in this one-sentence description of boosting: the number of iterations required, the computational cost of choosing good base classifiers, and the time and space complexity from maintaining a distribution over training examples. Warmuth, Liao, and Ratsch (2006) propose TotalBoost, which is a "totally corrective" boosting algorithm. Intuitively, on each round, totally corrective boosters choose base classifiers which give more information not present in previously chosen base classifiers. This leads to fewer iterations and smaller final hypotheses. Barutcuoglu, Long, and Servedio (2007) describe an alternative model for boosting where assumptions about the diversity of base classifiers allow the booster to learn in a single pass over the set of base classifiers. This eliminates the need to optimize over all base classifiers on each round. Bradley and Schapire (2007) propose an algorithm called FilterBoost which trains on examples drawn from an oracle rather than a fixed set of examples. This alternative learning framework can model learning via a single pass over the set of training examples and allows the booster to train efficiently on very large datasets.
Link this pageWould you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !