Assessing the Reliability and Validity of Multi-Attribute Utility Procedures: An Application of the Theory of Generalizability
Abstract
This report presents a theoretical rationale for assessing the reliability, validity, and dependability of multi-attribute utility models and techniques. If an investigator is advocating the use of a MAU model or procedure he or she is interested in generalizing from ovservations at hand to a universe or domain of observations that are members of that same universe. The universe must be unambiguously defined but it is not necessary to assume that universe as having any statistical properties such as uniform variance or covariances. A study of generalizability is conducted by taking measurements on persons, stimuli, tasks, etc. that are assumed to be randomly representative of a universe an investigator wishes to generalize to. The ratio of an estimate of the universe 'score' variance to an estimate of the observed score variance is the coefficient of generalizability. This is estimated by the intra-class correlation coefficient. ANOVA and the Expected Mean Square paradigm of Cornfield and Tukey is used to obtain the appropriate variance estimates. The theory dispenses with unnecessary and unwarranted assumptions, and eliminates the distinction between reliability and validity.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jul 01, 1975
- Accession Number
- ADA016282
Entities
People
- J. R. Newman
Organizations
- University of Southern California