A Simulation‐Based Method for Finding the Optimal Number of Options for Multiple‐Choice Items on a Test
Abstract
For a multiple‐choice test under development or redesign, it is important to choose the optimal number of options per item so that the test possesses the desired psychometric properties. On the basis of available data for a multiple‐choice assessment with 8 options, we evaluated the effects of changing the number of options on test properties (difficulty, reliability, and score comparability) using simulation. Using 2 criteria (low frequency and poor discrimination) to remove nonfunctioning options and 2 schemes (random and educated guessing) to model hypothetical response behavior for the removed options, we found that decreasing the number of options (from 8) created an easier test form but that a test form with reduced options could be more reliable if low‐discriminating options were removed and an educated guessing strategy were assumed. We present a rationale for the optimal number of options for this test being approximately 5, which would result in a shorter test while preserving its psychometric quality. Simulation methods discussed in this report could be applied to any test to compare the effects of changing the number of options.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Jun 14, 2018
- Source ID
- 10.1002/ets2.12209
Entities
People
- Hongwen Guo
- Jiyun Zu
- Patrick Kyllonen
Organizations
- Educational Testing Service
- Intelligence Advanced Research Projects Activity
- Office of the Director of National Intelligence