Evaluation and Visualization Tools for Crowdsourcing Games
Abstract
Crowdsourcing has been considered as a panacea for a number of computationally hard problems that involves using the crowd in combination with machine learning algorithms. With the success ofFoldlt (for protein folding) it has been imagined that most computational hard problems can be easily converted into "games" that could be used to engage the crowd. Crowd-Sourced Formal Verification (CSFV) is one such chosen program at DARPA In running the experiments it was observed that gamers were not engaged in the game after a few trials. The questions that this proposal will address, based on data collected during CSFV, are: (a) what are the design issues with the games that affects retention of players, and how can we advance current evaluation techniques to allow us to understand how to test and inspect for such retention issues with game designs early on in the development cycle, (b) what are the different strategies that game players formulate, are there any dominant strategies, unique strategies that can help model different solution patterns, (c) is the correlation between individual game players ability and their performance in a team game, and (d) how to develop metrics for assessment of team performance during problem-solving games. The proposed approach include the development of visualization tools for identifying different problem solving strategies, and measuring individual performance and team performance during a team game in the context of crowdsourcing. Secondly, and more importantly, the Offeror, based on data collected during CSFV exercises, will identify the reasons for chum in the players that participated in that crowdsourced game.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jan 12, 2017
- Source ID
- W911NF1510411
Entities
People
- Magy Seif El-Nasr
Organizations
- Army Contracting Command
- Defense Advanced Research Projects Agency
- Northeastern University