A Comparison of the Fairness of Adaptive and Conventional Testing Strategies.
Abstract
This report examines how selection fairness is influenced by the characteristics of a selection instrument in terms of its distribution of item difficulties, level of item discrimination, degree of item bias, and testing strategy. Computer simulation was used in the administration of either a conventional or Bayesian adaptive ability test to a hypothetical target population consisting of a minority and majority subgroup. Fairness was evaluated by three indices which reflect the degree of differential validity, errors in prediction (Cleary's model), and proportion of applicants exceeding a selection cutoff (Thorndike's model). Major findings are (1) when used in conjunction with either the Bayesian or conventional test, differential prediction increased fairness and facilitated the interpretation of the fairness indices; (2) the Bayesian adaptive tests were consistently fairer than the conventional tests for all item pools above the alpha=.7 discrimination level for tests of more than 30 items; (3) the differential prediction version of the Bayesian adaptive test produced almost perfectly fair performance on all fairness indices at high discrimination levels; and (4) the placement of subgroup prior distribution in the Bayesian adaptive testing procedure can affect test fairness. (Author)
Document Details
- Document Type
- Technical Report
- Publication Date
- Aug 01, 1978
- Accession Number
- ADA059436
Entities
People
- David J. Weiss
- Steven M. Pine
Organizations
- University of Minnesota