Generalizability Theory as Evidence of the Reliability and Validity of Work Sample Tests and Proficiency Ratings.

Abstract

This paper uses generalizability (G) theory as a framework to investigate the reliability and construct validity of the Air Force Job Performance Measurement System. G theory was a useful technique for examining the psychometric quality of the measurement system because it permits the specification and estimation of multiple sources of measurement error. G theory was applied to newly developed work sample procedures, Walk-Through performance Tests and to a series of job proficiency ratings comprised of four different rating forms collected from three rating sources. The results provided evidence of strong convergent and discriminant validity of these work sample tests, strong convergent validity across rating forms, and moderate discriminant validity of the rating system. However, ratings did not generalize across self, peer and supervisor rating sources. Results are discussed in terms of their practical and theoretical implications.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Dec 01, 1995
Accession Number
ADA317650

Entities

People

  • Kurt Kraiger
  • Mark S. Teachout

Organizations

  • University of Colorado Boulder

Tags

Communities of Interest

  • Human Systems

DTIC Thesaurus Topics

  • Air Force
  • Air Traffic
  • Analysis Of Variance
  • Applied Psychology
  • Computer Programs
  • Data Analysis
  • Governments
  • Human Resources
  • Information Science
  • Jet Engines
  • Management Personnel
  • Measurement
  • Performance Tests
  • Psychology
  • Reliability
  • Supervisors
  • Test And Evaluation

Readers

  • Instructional Design and Training Evaluation.