Generating the CausX TA1 Assessment Data
Abstract
In Spring 2020, IDA performed an assessment of DARPAs Causal Exploration (CausX) program. The CausX program involves extracting events, causal assertions, and related elements from natural language documents. IDAs assessment compared the extraction quality of four performers participating in CausX. The IDA team developed an approach to select each performers highest-quality work. To ensure highest-quality work was well-defined, the team created an automatable, repeatable process for selecting events and causal assertions. This document describes the process and how to perform it. It uses open source software and can be performed on a variety of modern computing platforms; IDA has executed it on Windows and Linux. All files necessary to execute the process are included in this deliverable. An analyst who wants toreproduce IDAs results can use these files and follow the process steps to obtain the same spreadsheet data (allowing for some variation due to randomization; steps involving randomization note the fact) IDA used in its assessment.
Document Details
- Document Type
- Technical Report
- Publication Date
- Nov 01, 2020
- Accession Number
- AD1148052
Entities
People
- Peter A. Kind
- Steven P. Wartik
- Susan K. Numrich
Organizations
- Institute for Defense Analyses