Rater Accuracy Study.
Abstract
Eight hundred eighty-two airmen were divided into more accurate and less accurate rating groups on the basis of their ability to estimate scores of their peers on a vocabulary test. To test whether the method actually did separate more accurate from less accurate raters, correlations were calculated within the more and less accurate groups between ratings of carefulness and scores on carefulness tests and between ratings of decisiveness and scores on decisiveness tests. The analysis consisted of counting the number of times the correlations between test scores and ratings in the more accurate group were larger than the analogous correlations in the less accurate group and computing the probability that this number of differences in the predicted direction might be expected by chance. It appears that this method of identifying accurate raters does work reasonably well. Several auxiliary questions concerning the best qualities to test in the estimating part of the study and the generalizability of rater accuracy to different personality characteristics were asked and answered. (Author)
Document Details
- Document Type
- Technical Report
- Publication Date
- Feb 01, 1979
- Accession Number
- ADA066779
Entities
People
- Cecil J. Mullins
- James A. Earles
- James Wilbourn
- Kevin Seidling
Organizations
- Brooks Air Force Base