test variable is defined (‘The mean change in the
pretreatment to posttreatment blood pressure’). (c)
A statistical hypothesis is formulated, that is the
scientific hypothesis is formulated in terms of the
test variable (‘The mean reduction in blood pres-
sure in the group treated with the new drug is
greater than that of the placebo-treated group’).
(d) A decision criterion is formulated. That is,
which outcomes of the experiment would lead to
the rejection and which to the acceptance of the
hypothesis. (e) The experiment is conducted, and
the data are collected and the test variable observed
in the experiment is calculated (‘The mean change
in pretreatment to posttreatment blood pressure is
calculated for the two groups of subjects’). (f) A
decision is made using the decision criterion.
The key difference between the statistical
method and the scientific method is that statisti-
cally the result, no matter how unlikely, is not
impossible. Therefore, any decision to confirm or
reject a hypothesis is liable to error. Two types of
error are possible, as summarized in Table 25.1.
25.3 The statistical test:
the null hypothesis, error
probabilities, statistical
powerSeemingly, the decision whether a drug is effica-
ciousor not,is adichotomy. In reality,however, itis
a continuum. If we consider the measurable effect
(E) of our drug in lowering diastolic blood pres-
sure, then lack of efficacy corresponds toE¼0.
Positive efficacy corresponds toE>0, which con-
tains a continuum of possibilities depending on the
magnitude of the effect. Thus, the hypothesis of no
efficacy is very specific in terms of the size of the
effect and is called asimple hypothesis, whereas a
hypothesis corresponding to a range of values is
called acomposite hypothesis.
As we have seen, both the scientific method
and the statistical method are designed to prove a
claim false rather than true. In drug testing, the
statistical experiment is designed to reject thenull
hypothesis– the hypothesis of lack of efficacy, or
that there is no difference between the treatments
being tested. Table 25.1 mentioned above is
applied to the null hypothesis.
Type I error: Rejection of the null hypothesis
when it is true (an ineffective drug is judged
effective).
Type II error: Acceptance of the null hypothesis
when it is false (an effective drug is judged inef-
fective).
Type I error is often also called ‘False Positive’
and type II error ‘False Negative’. Because rejec-
tion of the null hypothesis enables one to make the
scientific claim that the study was performed to
prove, statisticians label such a rejection assignifi-
cant. When the result of a test is declared signifi-
cant, the only error that could occur is type I error.
Clearly, the smaller the probability of type I error,
the more secure one is in rejecting the null hypoth-
esis. The probability of a type I error is called the
significance levelof the test and is denoted bya.
The probability of a type II error is denoted byb;
1 b is called the power of the test, often
expressed as percent. Thus, the power of the test
is the probability of rejecting the null hypothesis
when it is false. When the null hypothesis is that the
drug is not efficacious, thepoweris the probability
that the test would declare the drug as efficacious
when indeed it is so. The null hypothesis is usually
a simple hypothesis. Therefore,ais usually a sin-
gle number. The alternative to the null hypothesis,
however, is typically a composite hypothesis. In
our antihypertensive drug testing example, this
alternative was the whole regionE>0. In this
case, the value ofband the power 1bdepend
on the specific value ofE. Thus, it is meaningless to
talk about the power of a statistical test without
specifying the alternative for which it applies. In
our example, the power of the test atE¼10 is theTable 25.1 Type I and type II errors in statistical
decision making
DecisionReal state Accept hypothesis Reject hypothesis
Hypothesis Type I error
is true
Hypothesis Type II error
is false
25.3 THE STATISTICAL TEST: THE NULL HYPOTHESIS, ERROR PROBABILITIES, STATISTICAL POWER 315