Multiple Simultaneous Hypothesis Testing is a main issue in many areas of information extraction:
1. rule extraction,
2. validation of genes influence,
3. validation of spatio-temporal patterns extraction (e.g. in brain imaging),
4. other forms of spatial or temporal data (e.g. spatial collocation rule),
5. other multiple hypothesis testing. In all above frameworks, the goal is to extract patterns such that some quantity of interest is significantly greater than some given threshold.
1. in rule extraction, the goal typically is the extraction of rules with confidence, lift and support significantly higher than a given threshold;
2. in multiple hypothesis testing, the goal typically is the extraction of significant comparisons among various averages simultaneously;
3. in spatio-temporal patterns extraction, the goal typically is the extraction of smooth (spatio-temporal) subsets of (0,1) with correlation significantly higher than a given threshold.
Along these lines, a type I error is to extract an entity which does not satisfy the considered constraint while a type II error is to miss an entity which does satisfy the constraint. How to estimate, bound, or (even better !) reduce type I and type II errors are the goals of the proposed challenge.
VC-theory, empirical process and various approaches related to simultaneous hypothesis testings are fully relevant, as well as specific approaches, e.g. based on simulations, resamplings or probes.
The challenge consists in extending previous results to the field of simultaneous hypothesis testing, or proposing new results specifically related to this topic.