Air Force Civilian Potential Appraisal System: Development
Abstract
An appraisal system for use by the Air Force in ranking competitive position civilian promotion eligibles was developed to meet legal and operational requirements. Demographic data and supervisor and peer ratings were obtained from a 20,000 case target sample to establish a pool of possible ranking elements. Variable reduction procedures eliminated those elements which are redundant, showed greatest likelihood of creating adverse impact, or which appeared less defensible legally or psychometrically. The remaining variables were submitted to 12-member promotion data panels for each of 23 homogeneous job family clusters. Policy-capturing techniques resulted in regression-weighted promotion algorithms that use unique combinations of 19 behavioral dimensions, depending on the job family and supervisory or nonsupervisory classification of the job to be filled. Correlations between job-family-specific, promotion panel rankings and algorithm-developed rankings ranged from .92 to .98, indicating remarkably consistent panel member policies and high agreement between raters. Separate panels convened for two job families resulted in cross-validation correlations of .94 to .98 between their rankings indicating that the algorithms do not reflect just the judgment of panel members who happened to be selected but rather are indicative of the judgments of experienced members of an entire job family. A single supervisory rating form and 40 job-family-specific algorithms were recommended for operational implementation.
Document Details
- Document Type
- Technical Report
- Publication Date
- Nov 01, 1984
- Accession Number
- ADA148909
Entities
People
- D. E. Black
- R. B. Gould
- W. H. Cummings
Organizations
- Air Force Research Laboratory