Assessing and Expanding the Limits of Collective Intelligence
Abstract
The major goals for the project, expanded from the Statement of Work, are as follows: 1) Construct specialized optimization tools that are capable of aggregating and gleaning insights from continuous, ordinal, and multimodal judgement-elicitation data using different choices of distance functions. These models and algorithms will be equipped to solve problems with over 100 evaluation entities and agents efficiently (i.e., in the order of minutes). 2) Develop customized statistical models for generating nontrivial problem instances with objectively defineddegrees of collective similarity/dissimilarity from an underlying ground-truth set. The goal of the related tasks are to incorporate a variety of characteristics relevant to complex estimation tasks: incompleteness, sparsity, noise/error, imperfect subtask overlap, and high dimensionality. 3) Test the efficacy of the proposed aggregation methodologies, specifically to derive empirical parametric descriptions of the relationships of varying elicitation choices and statistical distribution configurations with the similarity between the aggregate estimate and the underlying ground truth; tests may also be performed on existing benchmarks for other applications; for example for wireless sensor network estimation in contested environments. 4) Validate the proposed tools in practice through the implementation of crowd based estimation tasks. These willbe guided by standard crowd wisdom benchmarks of varying difficulties such as ranking the difficulties of puzzles, counting randomly distributed dots in images, and/or box-office prediction. 5) Leverage multidisciplinary collaborations to demonstrate the practical value of the developed tools. 6) Prepare a final technical report summarizing the outcomes of the project. Prepare and submit manuscripts to peer-reviewed conferences and journals.
Document Details
- Document Type
- Technical Report
- Publication Date
- Apr 24, 2020
- Accession Number
- AD1113648
Entities
People
- Adolfo R. Escobedo
Organizations
- Arizona State University