Test Reliability and Experimental Power,
Abstract
The reliability coefficient, p(xx), has long been accepted as an index of the stability, repeatability, and precision of psychological tests. Because pxx measures the proportion of the variance in a set of scores attributable to variation among individuals, values of p(xx) are sometimes compared to justify using particular tests in studies of individual differences. Values of p(xx) are also sometimes compared to justify using particular tests in experimental research. The latter practice is usually justified by arguing that larger values of p(xx) imply greater measurement precision and, therefore, potentially greater sensitivity to experimental treatments. That argument is not generally correct because the individual variation measured by p(xx) is frequently confounded with measurement error in the denominators of significance tests. The effects of this confounding lead to 'paradoxical' situations in which reliability, as measured by p(xx), may be inversely related (or unrelated) to experimental precision, as measured-by the reciprocal of experimental error. Because the power of an experiment increases with precision, as just defined, conditions that invert or negate the relationship between p(xx) and precision also invert or negate the relationship between p(xx) and power. These considerations do not mean that the reliability coefficient is necessarily irrelevant to experimental research. Because experimental designs differ in the degree to which they are influenced by individual variation, a consideration of the value of p(xx) a specific test yields will sometimes provide information about the best design in which to use that test. Statistics, Psychometrics, Reliability, Power.
Document Details
- Document Type
- Technical Report
- Publication Date
- Dec 01, 1993
- Accession Number
- ADA279607
Entities
People
- R. R. Stanny
Organizations
- Naval Aerospace Medical Research Laboratory