Why do we Need Significance Levels.
Abstract
Significance tests are commonly used in many application areas as attempts to formally confirm or refute specific conclusions. For example, in the social sciences (e.g. psychology, sociology, and econometrics) there is often much more emphasis on data-fitting and seeking 'significant' results than on developing proper mathematical models which reltae in an inductively sensible way to the real-life problem. In the present paper a new formulation is used to demonstrate that significance tests tend to be much too ready to reject the null hypothesis for large sample sizes. It is recommended that the usual percentage points should be replaced by quantities depending in a particular way upon sample size, but not upon a choice of significance level. The phenomena discussed would appear to be particularly relevant to the area of scientific reporting. For example, many results in applied journals which might have been viewed as 'significant,' because they yield a low p-value, may in fact serve to detract from the very scientific theory which they claim to substantiate. For large sample sizes, the techniques proposed in this paper permit a larger range of viable null hypotheses than experienced under fixed-size significance testing. It should therefore be easier to use them to find a data-credible model which is also reasonable in real-life terms.
Document Details
- Document Type
- Technical Report
- Publication Date
- Oct 01, 1979
- Accession Number
- ADA079739
Entities
People
- Tom Leonard
Organizations
- University of Wisconsin–Madison