Appropriateness Measurement for Computerized Adaptive Tests
Abstract
The effects of an initial sequence of random responses to 15-, 20-, and 25-item adaptive tests were examined in a series of simulation studies. Random responding on as two items had a substantial effect on an examinee's score. Thus it is important to determine whether--as a result of carelessness, test anxiety, computer anxiety, failure to understand instructions, or other reasons--an examinee has answered the first several items haphazardly. It was shown by use of an optimal appropriateness index, the likelihood ratio (LR) index, that a large proportion of faulty test scores can be identified. The performance of LR was evaluated by determining hit rates and false positive rates in a series of studies concerning: (a) comparisons with other indices, (b) the use of a security procedure during item selection for the adaptive test, (c) standardization, and (d) misspecification of the number of items with random answers. The LR index detected initial sequences of random responses with high accuracy with and without a security procedure during item selection. Other appropriateness indices were considerably less accurate. Standardization greatly decreased the power of LR at low false positive rates. Finally, misspecification of the length of the initial segment of random responses systematically reduced the power of the LR index to detect aberrance. Keywords: Appropriateness measurement; Aptitude tests; Computerized adaptive test.
Document Details
- Document Type
- Technical Report
- Publication Date
- Dec 01, 1989
- Accession Number
- ADA216121
Entities
People
- Gregory L. Candell
- Michael V. Levine
Organizations
- University of Illinois Urbana–Champaign