Measuring Air Traffic Controller Performance in a High-Fidelity Simulation
Abstract
In the summer of 1997, the Air Traffic Selection and Training (AT-SAT) High Fidelity Simulation Study was conducted at the FAA Academy in Oklahoma City, OK. The purpose of the study was to test the performance of 107 operational en route controllers during 2 1/2 days of simulations. The performance of these controllers during the high- fidelity simulations was compared with their performance on two medium-fidelity performance measures to assess the construct validity of the latter measures to serve as criteria against which to validate a set of selection tests. The reports included in this document describe the high-fidelity simulation exercise, the development of performance measures utilized during the exercise, and the interrelationships between the performance measures. The first report describes the development of a work sample approach to capturing air traffic controller performance, and establishes that high fidelity performance measures can adequately reflect the performance of the controller. The work sample was developed in an environment that simulated as nearly as possible the actual conditions existing in the controller's job, but was conducted in a "generic" airspace. Scenario development included the most important tasks from the task-based job analysis developed for the AT-SAT project. Sufficient time was provided for participating controllers to learn the airspace and procedures and demonstrate their knowledge through 1) a multiple choice test of airspace% knowledge and 2) running 8 practice scenarios. Performance was measured by 1) an over-the-shoulder (OTS) rating scale, 2) counts of mistakes, 3) counts of actions that would be required to move aircraft from the sector at the end of the scenario, and 4) statistics derived from aircraft positions and controller/pilot data entries recorded for the simulation. The second report used measures collected during the high-fidelity simulation study to predict the overall OTS performance rating.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 01, 2000
- Accession Number
- ADA373813
Entities
People
- Carol A. Manning
Organizations
- Federal Aviation Administration