en
0.25
0.5
0.75
1.25
1.5
1.75
2
Learning from Inconsistent and Unreliable Annotators by a Gaussian Mixture Model and Bayesian Information Criterion
Published on 2011-11-303803 Views
Supervised learning from multiple annotators is an increasingly important problem in machine leaning and data mining. This paper develops a probabilistic approach to this problem when annotators are n
Related categories
Presentation
Learning from Inconsistent and Unreliable Annotators00:00
Typical supervised classification01:24
Golden ground truth01:40
Subjective ground truth from multiple annotators01:53
Annotations from multiple annotators03:00
We are interested in03:38
How to judge an annotator? - 103:59
How to judge an annotator? - 204:31
Varying effectiveness on types of data05:40
How to approximate the distribution of the instances?06:09
Problem Statement06:51
If we know the true labels07:09
How to find the unknown true labels - 108:06
How to find the unknown true labels - 208:30
GMM-MAPML Algorithm08:47
Analysis of the model10:00
Emotional speech classification10:46
Dataset: EMA database from University of Southern California11:29
Experiment Results: ROC comparisons12:57
Experiment Results: GMM-MAPML based estimates of annotators’ accuracy14:47
Protein Disorder Prediction15:45
CASP9 Disorder Dataset18:31
CASP9 Assessment Scores20:48
GMM-MAPML based estimates of CASP9 disorder predictors’ accuracy22:19
Thank you22:47