Crowd Sourcing Data Collection through Amazon Mechanical Turk

Abstract

Crowdsourcing is an increasingly popular technique used to complete complex tasks or collect large amounts of data. This report documents the effort to employ crowdsourcing using the Mechanical Turk service hosted by Amazon. The task was to collect labeling data on several thousands of short videos clips as such labels would be perceived by a human. The approach proved to be viable, collecting large amounts of data in a relatively short time frame, but required specific considerations for the population of workers and impersonal medium through which data were collected.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Sep 01, 2013
Accession Number
ADA585850

Entities

People

  • Cynthia Pierce
  • Nicholas Fung

Organizations

  • United States Army Research Laboratory

Tags

Communities of Interest

  • Materials and Manufacturing Processes

DTIC Thesaurus Topics

  • Crowdsourcing
  • Data Sets
  • Data Storage Systems
  • Databases
  • English Language
  • Grammars
  • Graphical User Interface
  • Html
  • Information Retrieval
  • Information Science
  • Language
  • Linguistics
  • Markup Languages
  • Recognition
  • United States
  • User Interface
  • Xml

Fields of Study

  • Computer science

Readers

  • Agent-Based Social Robotics and Mobile-Assisted Learning in Virtual Environments.
  • Educational Psychology
  • Geospatial Intelligence and Artificial Intelligence Analytics