Superforecasting: Proven Practices for Leveraging Human Ingenuity
Abstract
Those of us who work for the military in some capacity are well aware of the emphasis placed on lessons learned. Army's recognition of the importance of learning from past experience is embodied in the Center for Army Lessons Learned (CALL). CALL identifies, collects, analyzes, disseminates, and archives lessons and best practices ... to facilitate the Army's ... adaptation to win wars. There is great wisdom in the practice of reflecting on our experiences for building a better future in a complex world. When we truly learn a lesson, we incorporate it into our practices to advance our knowledge and capability, and to improve our simulation products. But what of lessons unlearned, those things we have tripped over, documented, forgotten and thus have tripped over again. Is there a role for them? What about our failures, the ones we hesitate to celebrate in papers and presentations? Are we neglecting a valuable resource? Enter the notion of superforecasting. In 2010, the Intelligence Advanced Research Projects Agency (IARPA) issued a Broad Agency Announcement (BAA) entitled Aggregative Contingent Estimation (ACE) with the goal of dramatically enhancing the accuracy, precision and timeliness of intelligence forecasts for a wide range of event types. Among the participants, a newly developed program, the Good Judgment Project (GJP), aimed at harvesting the "wisdom of the crowd" while simultaneously examining the performance of participating individuals. About 2 percent of the 250 individuals in the "crowd" emerged as superforecasters who beat the benchmarks by as much as 30 percent. That result would be of little interest, except that superforecasting capability can be trained. The thrust of this paper is an examination of how the thought patterns for superforecasters could influence how we work as program managers, technologists and trainers to improve our products and perhaps contribute to training more effective, agile military leaders.
Document Details
- Document Type
- Technical Report
- Publication Date
- Mar 02, 2020
- Accession Number
- AD1222375
Entities
People
- S. K. Numrich
Organizations
- Institute for Defense Analyses