Learning with Millions of Examples and Dimensions - Competition proposal

author: Sören Sonnenburg, Intelligent Data Analysis Group, Fraunhofer Institute for Intelligent Analysis and Information Systems
published: Feb. 1, 2008,   recorded: December 2007,   views: 3682


Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


Over the years many different classification methods have been proposed in machine learning. However it is currently very difficult to judge which method is the most efficient with respect to training time and memory requirements and classification performance, which are the practically relevant criteria. A possible explanation for this difficulty is that methods are (often) evaluated under different conditions: For instance different datasets, evaluation criteria, model parameters and stopping conditions are used. We would therefore like to organize a competition, that is designed to be fair and enables a direct comparison of current large scale classifiers. To this end we plan to provide a generic evaluation framework tailored to the specifics of the competing methods, for example for Support Vector Machine classifiers, one would in addition to test-error record the objective value of the primal problem. Providing a wide range of datasets, each of which having specific properties, like extremely sparse, dense, high or low dimensional, we propose to evaluate the methods based on the following figures: training time vs. test error, dataset size vs. test error and dataset size vs. training time. We seek help from the community to gather relevant large-scale real-world data sets and to critically review and discuss fair evaluation criteria and finally invite researchers to co-organize and to participate in this challenge.

See Also:

Download slides icon Download slides: eml07_sonnenburg_lme_02.pdf (219.5 KB)

Help icon Streaming Video Help

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: