Calibrating Item Parameters Using Small Samples: A Comparison of Methods Using a Foreign Language Assessment
Abstract
In response to an expressed need to reduce the number of examinees used for field testing multiple-choice items, this study compares various calibration models thought to improve accuracy of item parameter estimates for small examinee sample sizes. Calibration methods range from fully estimated three-parameter logistic models to fully restricted approaches. Results of this study revealed that using prior information about both difficulty and discrimination of items enhanced the accuracy to which item parameter estimates were recovered, but not enough to adequately recover item characteristic curves or their parameters. However, test characteristic curves were recovered with moderate accuracy under small sample size conditions. It is recommended that no fewer than 100 examinees be used for field testing multiple-choice items and constructing parallel forms. Item-specific priors should be used during item calibration. Small sample sizes should not be used for obtaining item parameter estimates for computer adaptive testing environments.
Document Details
- Document Type
- Technical Report
- Publication Date
- Aug 30, 2023
- Accession Number
- AD1209466
Entities
People
- Tia M. Fechter
- W. A. Nicewander