Interpreting Graphs in a Wisdom of Crowds Forecasting Interface for Intelligence Analysts
Abstract
Intelligence analysts must integrate highly dynamic information in order to solve problems reactively and make predictions about future events proactively. Individual predictions may or may not be highly accurate, depending on the expertise of the forecaster. However, previous research on the wisdom of crowds has determined that often, aggregated estimates of multiple experts is even closer to the truth than most single expert forecasts. We have developed a tool as a means of aggregating and displaying the wisdom of crowds for Intelligence, Surveillance, and Reconnaissance (ISR) analysts. This tool allows analyst forecasters to make their own predictions regarding self-selected questions pertinent to mission objectives. Additionally, the tool displays the aggregate predictions of other analysts over a 1-month period, along with an individual analysts reasoning for each of their predictions. This allows any given analyst to consult the wisdom of the crowd and the opportunity to update their predictions, if desired. In order to ensure this tool is maximally effective and interpreted properly by analysts, it is critical that the graphical display of the crowds prediction information is comprehensible. We tested a variety of graph types (two forms of line graph, a bar graph, a box plot, and a dot plot) to display the wisdom of crowds with varying features to ascertain individuals' ability to accurately understand the presented information using a small pilot sample of participants
Document Details
- Document Type
- Technical Report
- Publication Date
- Oct 01, 2020
- Accession Number
- AD1143844
Entities
People
- Mary E. Frame
- Michaela Schwing