On Measuring the Effectiveness of Large-Scale Training Simulations
Abstract
This paper identifies and evaluates methods for measuring the benefit or utility of large-scale training simulations (LSTSs). The intended audience is those people who are directly responsible for assessing the training effectiveness of LSTSs. We examine literature related to the measurement of LSTS training effectiveness INCLUDING GUIDANCE PROVIDED BY THE INDIVIDUAL MILITARY Services and the Office of the Secretary of Defense (OSD), the body of research on performance measurement, and research practices related to experimental design and analysis. We then synthesize the findings from these literature sources to provide seven recommendations for performing training effectiveness analyses of LSTSs: (1) identify specific measurement issues, (2) devise a measurement plan that specifies performance measurement and research design, (3) use valid and reliable performance measures, (4) obtain the most valid measure of proficiency possible within the constraints of the evaluation, (5) to the extent possible and reasonable, impose experimental control on the research situation, (6) use analytic models in early stages and continue using these models throughout system development, (7) and include user reactions as an adjunct to performance data and analysis.
Document Details
- Document Type
- Technical Report
- Publication Date
- Oct 01, 2000
- Accession Number
- ADA394491
Entities
People
- Colin Hammon
- John E. Morrison
Organizations
- Institute for Defense Analyses