Identification and Evaluation of Simulator System Deficiencies
Abstract
The importance of fidelity in simulation training has been established as a key factor in training effectiveness. Design and development of simulation systems is dealt with commonly as an iterative procedure in which the goal is to improve on the training effectiveness of the system. Improvements are made based on methods of identifying and resolving deficiencies. Improvements must then be considered relative to available resources to determine how to prioritize the multiple options. The current paper presents a flexible methodology for evaluating the relative effects of multiple deficiencies on a simulation system, utilizing information from warfighters about the effectiveness of a Deployable Tactical Trainer (DTT). First, subject matter experts were asked to evaluate the fidelity of the current DTT system across a set of 199 experiences and emergency procedures (EP). Next, the same subject matter experts identified deficiencies of the current system. Then the warfighters evaluated the 199 experiences and EPs for which deficiency had the most adverse affect on training. A composite score was computed for each deficiency, using weighted variables accounting for (a) the amount of improvement possible and (b) the importance of training for each experience. Deficiency scores provide a means of comparing the relative effect of each deficiency on warfighter training. The impact of utilizing multiple sources of information is presented through a comparison of the different decisions that would result from using partial information or alternative weighting methods. The proximal implications of using the proposed methodology to have the greatest impact on improvement are discussed. The distal implications of the impact the improvements have on pilot perceptions, and ultimately, on objective pilot performance measures, are also discussed.
Document Details
- Document Type
- Technical Report
- Publication Date
- Nov 01, 2007
- Accession Number
- ADA501452
Entities
People
- Brian T. Schreiber
- Justin H. Prost
- Winston Bennett Jr.