An Introduction to Crowdsourcing for Language and Multimedia Technology Research
edited by: Maristella Agosti, Nicola Ferro, Pamela Forner, Henning Müller, Giuseppe Santucci
Language and multimedia technology research often relies on large manually constructed datasets for training or evaluation of algorithms and systems. Constructing these datasets is often expensive with significant challenges in terms of recruitment of personnel to carry out the work. Crowdsourcing methods using scalable pools of workers available on-demand offers a flexible means of rapid low-cost construction of many of these datasets to support existing research requirements and potentially promote new research initiatives that would otherwise not be possible.