Aiding the Intelligence Analyst in Situations of Data Overload: A Simulation Study of Computer-Supported Inferential Analysis under Data Overload
Abstract
A simulation study of inferential analysis was conducted with ten professional intelligence analysts. Using a process tracing methodology, patterns in vulnerabilities were identified when analysts were asked to analyze something outside their base of expertise, were tasked with a tight deadline, and had a large data set. Study participants were vulnerable to missing critical information. All the participants were observed to use relatively primitive search tactics, quickly narrowing in on a set of documents through the addition of keywords to an initial query. All of the participants missed some of the nine documents that were categorized as high quality. A group of four participants who found and relied upon some of the high quality documents took more time, read more documents, and made fewer inaccurate statements in their verbal briefings than a group of four participants who did not. In addition, three sources of inaccurate statements were identified. First, study participants sometimes relied upon assumptions that would normally be correct, but did not apply in this situation. Second, participants sometimes repeated information that was inaccurate in a document that they had read. Third, participants were observed to rely upon information that was considered accurate at one point in time, but then was later overturned in subsequent updates.- The main contribution from this research was a model of potential vulnerabilities in inferential analysis under challenging conditions. These vulnerabilities are informative because they point to a set of challenging design criteria that human-centered solutions to data overload need to meet.
Document Details
- Document Type
- Technical Report
- Publication Date
- May 01, 1999
- Accession Number
- ADA395332
Entities
People
- David D. Woods
- Emilie M. Roth
- Emily S. Patterson
Organizations
- Ohio State University