en
0.25
0.5
0.75
1.25
1.5
1.75
2
Learning from Inconsistent and Unreliable Annotators by a Gaussian Mixture Model and Bayesian Information Criterion
Published on Nov 30, 20113796 Views
Supervised learning from multiple annotators is an increasingly important problem in machine leaning and data mining. This paper develops a probabilistic approach to this problem when annotators are n
Related categories
Chapter list
Learning from Inconsistent and Unreliable Annotators00:00
Typical supervised classification01:24
Golden ground truth01:40
Subjective ground truth from multiple annotators01:53
Annotations from multiple annotators03:00
We are interested in03:38
How to judge an annotator? - 103:59
How to judge an annotator? - 204:31
Varying effectiveness on types of data05:40
How to approximate the distribution of the instances?06:09
Problem Statement06:51
If we know the true labels07:09
How to find the unknown true labels - 108:06
How to find the unknown true labels - 208:30
GMM-MAPML Algorithm08:47
Analysis of the model10:00
Emotional speech classification10:46
Dataset: EMA database from University of Southern California11:29
Experiment Results: ROC comparisons12:57
Experiment Results: GMM-MAPML based estimates of annotators’ accuracy14:47
Protein Disorder Prediction15:45
CASP9 Disorder Dataset18:31
CASP9 Assessment Scores20:48
GMM-MAPML based estimates of CASP9 disorder predictors’ accuracy22:19
Thank you22:47