The Development of Training Programs to Increase Accuracy with Different Rating Formats.
Abstract
The research investigated potential interactive effects of format and training on rating accuracy. Training was developed in accordance with the cognitive demands placed on raters by each of two rating formats: behaviorally anchored rating scales (BARS) and behavioral observation scales (BOS). In order to investigate format x training interactions, the effects of each training procedure were assessed for both types of scales. In addition to increasing accuracy, congruent training (BARS training with BARS and BOS training with BOS) was hypothesized to increase interrater agreement. One hundred and forty-four subjects were randomly assigned to 1 of 6 cells defined by combinations of training (BARS, BOS, control) and format (BARS and BOS). Analyses of variance revealed that for those using a BARS format, ratings from the BARS training group were more accurate than ratings from the BOS training group, which, in turn, were more accurate than ratings from the control group. When BOS were used to rate performance, only BOS training led to higher rater accuracy. Interrater agreement was greater in congruent training groups. Implications, limitations, and future research directions are discussed. (Author)
Document Details
- Document Type
- Technical Report
- Publication Date
- May 01, 1984
- Accession Number
- ADA142009
Entities
People
- E. D. Pulakos
Organizations
- Michigan State University