Modeling annotator expertise: Learning when everybody knows a bit of something

Supervised learning from multiple labeling sources is an increasingly important problem in machine learning and data mining. This paper develops a probabilistic approach to this problem when annotator