0.25
0.5
0.75
1.25
1.5
1.75
2
Crowd Truth: Machine-Human Computation Framework for Harnessing Disagreement in Gathering Annotated Data
Published on Dec 19, 20142323 Views
In this paper we introduce the CrowdTruth open-source software framework for machine-human computation, that implements a novel approach to gathering human annotation data for a variety of media (e.g.
Related categories
Chapter list
CrowdTruth.org00:00
Importance of Human Annotation00:35
Human Disagreement01:59
disagreement can reflect the degree of clarity in a sentence02:14
disagreement can indicate ambiguity of the relti7on03:49
disagreement can indicate low quality workers05:12
CrowdTruth SoIware Components: Machines & Crowds Workflow06:01
CrowdTruth SoIware07:39
CrowdTruth SoIware: Crowdsourcing Job Analytics08:17
CrowdTruth SoIware: Worker Analytics09:12
crowdtruth.org github.com/CrowdTruth10:13