Counteracting estimation bias and social influence to improve the wisdom of crowds
Abstract
Aggregating multiple non-expert opinions into a collective estimate can improve accuracy across many contexts. However, two sources of error can diminish collective wisdom: individual estimation biases and information sharing between individuals. Here, we measure individual biases and social influence rules in multiple experiments involving hundreds of individuals performing a classic numerosity estimation task. We first investigate how existing aggregation methods, such as calculating the arithmetic mean or the median, are influenced by these sources of error. We show that the mean tends to overestimate, and the median underestimate, the true value for a wide range of numerosities. Quantifying estimation bias, and mapping individual bias to collective bias, allows us to develop and validate three new aggregation measures that effectively counter sources of collective estimation error. In addition, we present results from a further experiment that quantifies the social influence rules that individuals employ when incorporating personal estimates with social information. We show that the corrected mean is remarkably robust to social influence, retaining high accuracy in the presence or absence of social influence, across numerosities and across different methods for averaging social information. Using knowledge of estimation biases and social influence rules may therefore be an inexpensive and general strategy to improve the wisdom of crowds.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Apr 01, 2018
- Source ID
- 10.1098/rsif.2018.0130
Entities
People
- Albert B Kao
- Andrew M. Berdahl
- Andrew T Hartnett
- Christos C. Ioannou
- Iain Couzin
- Joseph Bak-Coleman
- Matthew Lutz
- Xingli Giam
Organizations
- Argo AI
- Army Research Office
- Division of Integrative Organismal Systems
- Harvard University
- Human Frontier Science Program
- James S. McDonnell Foundation
- John Templeton Foundation
- Max Planck Institute for Ornithology
- National Science Foundation Directorate for Mathematical & Physical Sciences
- Natural Environment Research Council
- Office of Naval Research
- Princeton University
- Santa Fe Institute
- University of Bristol
- University of Konstanz
- University of Tennessee
- University of Washington