Learning from Inconsistent and Unreliable Annotators by a Gaussian Mixture Model and Bayesian Information Criterion thumbnail
Pause
Mute
Subtitles
Playback speed
0.25
0.5
0.75
1
1.25
1.5
1.75
2
Full screen

Learning from Inconsistent and Unreliable Annotators by a Gaussian Mixture Model and Bayesian Information Criterion

Published on Nov 30, 20113794 Views

Supervised learning from multiple annotators is an increasingly important problem in machine leaning and data mining. This paper develops a probabilistic approach to this problem when annotators are n

Related categories

Chapter list

Learning from Inconsistent and Unreliable Annotators00:00
Typical supervised classification01:24
Golden ground truth01:40
Subjective ground truth from multiple annotators01:53
Annotations from multiple annotators03:00
We are interested in03:38
How to judge an annotator? - 103:59
How to judge an annotator? - 204:31
Varying effectiveness on types of data05:40
How to approximate the distribution of the instances?06:09
Problem Statement06:51
If we know the true labels07:09
How to find the unknown true labels - 108:06
How to find the unknown true labels - 208:30
GMM-MAPML Algorithm08:47
Analysis of the model10:00
Emotional speech classification10:46
Dataset: EMA database from University of Southern California11:29
Experiment Results: ROC comparisons12:57
Experiment Results: GMM-MAPML based estimates of annotators’ accuracy14:47
Protein Disorder Prediction15:45
CASP9 Disorder Dataset18:31
CASP9 Assessment Scores20:48
GMM-MAPML based estimates of CASP9 disorder predictors’ accuracy22:19
Thank you22:47