Comparison of Live and Simulated Adaptive Tests
Abstract
The purpose of this research was to compare test scores obtained from live administration of several adaptive tests with those scores obtained from the same sample of individuals through computer simulation of the adaptive tests. The use of computer simulation techniques for the evaluation of adaptive testing protocols has gained wide usage. However, the validity of these simulation techniques has not been established. Three adaptive testing procedures were implemented, using two distinct item types. The adaptive testing procedures included (a) Two-Stage Adaptive Test, in which a 10-item routing test was followed by one of five 30-item measurement tests; (b) Pyramidal Adaptive Test, in which the subject was branched through a pyramidal structure of items until a total of 10 items were administered; and (c) Stratified Adaptive Test, in which items were selected from nine pools of items stratified on item difficulty. The two item types used were Word Knowledge and Visual Scanning. Approximately 400 subjects were tested on each type of item. In addition to the three adaptive tests, each subject was also tested using a 220-item test in a conventional format. Item responses for each subject from the conventional test administration were used to generate simulated test scores for each of the three adaptive tests. Results indicated that for both types of items, the simulated tests are not strictly parallel forms of the live tests. It was concluded that caution must be exercised in the use of computer simulation and that results from such procedures are not completely generalizable to live testing situations; however, the practical use of simulation was supported.
Document Details
- Document Type
- Technical Report
- Publication Date
- Dec 01, 1982
- Accession Number
- ADA124167
Entities
People
- David R. Hunter
Organizations
- Air Force Research Laboratory