A Bayesian Method for Evaluating Trainee Proficiency
Abstract
In any testing or evaluation program, there will be some percentage of false positives and false negatives, i.e., misclassifications will occur. A decisionmaker therefore needs to make a best estimate about the true level of proficiency of an examinee. A multiparameter, programmable model was developed to examine the interactive influence of certain parameters on the probability of deciding that an examinee had attained a specified degree of mastery through a program of instruction. The parameters, readily obtainable from decisionmakers, include: (1) the number of assumed mastery states ('master,' 'intermediate,' 'nonmaster'), (2) the prior distribution of scores from similar examinee groups, and (3) the number of test trials or items that could be given. Results of several simulations showed that the degree of confidence that a decisionmaker can have in his decision (e.g., 'x%' certainty that an examinee is a master) is markedly affected by values for the abovementioned parameters. A key feature of a Bayesian model is that testing time, manpower, expense, and test length can be reduced if the 'prior' information is accurate and valid for the particular tested group. If not, little can be gained from a Bayesian model. Simulated test results also showed that a test can be too short to be of any decisionmaking value.
Document Details
- Document Type
- Technical Report
- Publication Date
- Sep 01, 1978
- Accession Number
- ADA062245
Entities
People
- Frederick H. Steinheiser Jr.
- Kenneth I. Epstein
Organizations
- U.S. Army Research Institute for the Behavioral and Social Sciences