The Development and Evaluation of a Behaviorally Based Rating Form for Assessing Air Traffic Controller Performance.
Abstract
The evaluation of air traffic controller performance is a complex process. While there are standard forms in field use, there is currently no comprehensive system for reliable observer evaluation. This research involves the development of a new form along with a training package for use in research and possibly operational testing. The form consists of 24 rating scales. These scales focus on observable actions that trained air traffic control specialists could identify to make behaviorally based ratings. The study evaluates the reliability of the rating form by determining the consistency of ratings obtained from six observers who viewed videotapes of a previously recorded simulation study. These observers were supervisors and training staff specialists from Terminal Radar Approach Control facilities nationwide. Prior to making formal ratings, the observers participated in a training program designed to help them become proficient in observational rating. During the evaluation phase of the study, the observers viewed 20 one hour videotapes of controllers working different traffic scenarios. The results indicated that most of the rating scales had reasonable inter-rater reliabilities ranging from r=.7 to r=.9. The study also identified the performance areas that were more difficult for observers to evaluate consistently, possibly due to misunderstanding rating criteria or overlooking critical controller actions.
Document Details
- Document Type
- Technical Report
- Publication Date
- Feb 01, 1997
- Accession Number
- ADA324727
Entities
People
- Earl S. Stein
- Randy L. Sollenberger
- Stan Gromelski
Organizations
- Federal Aviation Administration