INCOMMANDS TDP: Measures of Performance for the Threat Evaluation Capability
Abstract
In this document, we are concerned with the Threat Evaluation (TE) capability, developed for the INCOMMANDS TDP to support the operator involved in the shipboard TE process. In particular, we focus on evaluating the recommendations provided by this capability as well as evaluating the final TE decisions made by the operator when using this capability. A conceptual model explaining how these evaluations will be done is proposed. First, we propose a set of measures of performance (MoPs) for the assessment of the quality of the recommendations provided by the TE capability. These measures are: measures of accuracy and agreement, measures of timeliness, sensitivity analysis, input degradation analysis and anytime behaviour analysis. Second, we consider that the assessment of the level of cognitive activity that the human operator is experiencing is important. This evaluation will assess if higher quality of decision making is reached while the operator is overwhelmed or not. To do so, we propose the use of the following human factors metrics: situation awareness, workload and task relevant behaviour. Third, the quality of the TE decisions made by the operator (when using the capability) will be measured using complementary MoPs such as accuracy, timeliness and consistency. When analyzed together, these measures provide converging evidence that decision making quality has either increased or decreased.
Document Details
- Document Type
- Technical Report
- Publication Date
- Dec 01, 2009
- Accession Number
- ADA640031
Entities
People
- A. Benaskeur
- A. Frini
Organizations
- DRDC Valcartier