5 Introduction to Statistical Tests of Significance
This chapter covers the most used and yet most misunderstood statistical pro-
cedures, calledtestsortests of significance. The reason for the misunderstand-
ing is simple: language. The colloquial meaning of the wordtestis one of
no-nonsense objectivity. Students take tests in school, hospitals draw blood to
be sent to laboratories for tests, and automobiles are tested by the manufac-
turer for performance and safety. It is thus natural to think that statistical tests
are the ‘‘objective’’ procedures to use on data. The truth is that statistical tests
are no more or less objective than any other statistical procedure, such as con-
fidence estimation (Chapter 4).
Statisticians have made the problem worse by using the wordsignificance,
another word that has a powerful meaning in ordinary, colloquial language:
importance. Statistical tests that result in significance are naturally misunder-
stood by the public to mean that the findings or results are important. That’s
not what statisticians mean; it only means that, for example, the di¤erence they
hypothesized wasreal.
Statistical tests are commonly seriously misinterpreted by nonstatisticians,
but the misinterpretations are very natural. It is very natural to look at data
and ask whether there is ‘‘anything going on’’ or whether it is just a bunch of
meaningless numbers that can’t be interpreted. Statistical tests appeal to inves-
tigators and readers of research for a reason in addition to the aforementioned
reasons of language confusion. Statistical tests are appealing because they seem
to make adecision; they are attractive because they say ‘‘yes’’ or ‘‘no.’’ There
is comfort in using a procedure that gives definitiveanswersfrom confusing
data.
One way of explaining statistical tests is to use criminal court procedures
188