A Support Vector Method for Multivariate Performance Measures

author: Thorsten Joachims, Department of Computer Science, Cornell University
published: April 12, 2007,   recorded: August 2005,   views: 6640

See Also:

Download slides icon Download slides: student3_calibration.ppt (7.2 MB)

Help icon Streaming Video Help

Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


We examine the relationship between the predictions made by different learning algorithms and true posterior probabilities. We show that maximum margin methods such as boosted trees and boosted stumps push probability mass away from 0 and 1 yielding a characteristic sigmoid shaped distortion in the predicted probabilities. Models such as Naive Bayes, which make unrealistic independence assumptions, push probabilities toward 0 and 1. Other models such as neural nets and bagged trees do not have these biases and predict well calibrated probabilities. We experiment with two ways of correcting the biased probabilities predicted by some learning methods: Platt Scaling and Isotonic Regression. We qualitatively examine what kinds of distortions these calibration methods are suitable for and quantitatively examine how much data they need to be effective. The empirical results show that after calibration boosted trees, random forests, and SVMs predict the best probabilities.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Reviews and comments:

Comment1 Urvang, February 12, 2009 at 2:35 p.m.:

The title of this video and the content seem totally unrelated.

Comment2 Daniel , January 25, 2014 at 7:04 p.m.:

The attached presentations seems to be wrong...

Write your own review or comment:

make sure you have javascript enabled or clear this field: