Assessing Interrater Agreement in Job Analysis Ratings

Abstract

The present study explored the reliability of quantitative job analysis scores derived from detailed narrative job descriptions. Descriptions of 25 different jobs in a medium-sized military hospital were scored by four trained raters using the Position Analysis Questionnaire. Interrater agreement was assessed using a number of common indices. These indices suggested general agreement on the ratings, but differences were noted in the estimates produced. Reasons for such differences were explored and recommendations were made for avoiding potential difficulties in assessing interrater agreement on job analysis ratings. (AUTHOR)

Open PDF

Document Details

Document Type
Technical Report
Publication Date
May 22, 1981
Accession Number
ADA108994

Entities

People

  • Allan P. Jones
  • Debbi Main
  • Lee A. Johnson
  • Mark C. Butler

Organizations

  • Naval Health Research Center

Tags

Communities of Interest

  • Biomedical
  • Human Systems

DTIC Thesaurus Topics

  • Agreements
  • Analysis Of Variance
  • Applied Psychology
  • Biomedical Research
  • Classification
  • Coefficients
  • Data Science
  • Experimental Design
  • Information Science
  • Job Analysis
  • Mental Processes
  • Military Hospitals
  • Psychology
  • Ratings
  • Reliability
  • Test And Evaluation
  • Training

Fields of Study

  • Psychology

Readers

  • Computational Modeling and Simulation
  • Instructional Design and Training Evaluation.