Reweighting AT-SAT to Mitigate Group Score Differences
Abstract
The Air Traffic Selection and Training (AT-SAT) test battery is the selection tool for applicants for Air Traffic Control Specialist (ATCS) positions within the FAA who have not previously been employed as an air traffic controller. AT-SAT is an aptitude test developed to predict the likelihood of successfully learning ATCS skills. Before operational use, however, concerns were raised about the low passing rate of incumbent ATCS personnel who participated in the initial research and score differences between groups, which could result in possible unfair discrimination. To address these concerns, the subscores of AT-SAT were reweighted, and the additive constant was changed to yield a new total score. This study compares the original and new scoring methods using data from 724 developmental ATCSs who volunteered to take the AT-SAT. An average increase of 4.86 points was found with the new scoring method; the notional passing rate (achieving a score >/= 70) changed from 58.8% to 80%. American Indian/Alaskan Native, Hispanic, and black participants showed the greatest average increase in overall scores (i.e., 6.97, 6.98, and 7.02, respectively). The increase in scores of Hispanic and black participants was significantly higher than the increase in scores for white participants [F(4,689) = 6.186, p < .001]. However, a chi square analysis showed no differences between groups for the participants whose failing score with the original scoring method changed to a passing score with the new method. A Spearman rank correlation coefficient of .85 was found between the two scoring methods, indicating that the ranking of individual participants did not change significantly. No differences were found between groups in rank ordering of the two scoring methods, and no significant gender differences were found between the scoring methods. The study found that the new weighting formula has benefited all groups and is likely to reduce the potential of adverse impact.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jul 01, 2006
- Accession Number
- ADA461242
Entities
People
- Andrew R. Dattel
- Raymond E. King
Organizations
- Federal Aviation Administration