why anyone should not use a cutoff point other than
the customary 0.05 if he or she feels it is more
appropriate. But as in any other situation when
one deviates from a standard, one must explain the
reasons for doing sobeforethe experiment is per-
formed and before the data are known.
The statistical testing setup, as we have already
seen, is geared toward the declaration of statistical
significance. When a test is significant, we draw a
conclusion about the cause of the effect of interest.
If we decide to reject the null hypothesis, the
p-value is the type I error associated with this
decision. Therefore, the level of confidence in the
correctness of the decision depends on thep-value;
the smaller thep-value, the more confident one is
that the decision is correct.
What if the statistical test is not statistically
significant? If one accepts the null hypothesis
in this case, the error to be concerned about is the
type II error (see Table 25.1). At the design stage of
the trial, the statistician usually ascertains that the
test to be employed at the end has desired power at
clinically important alternatives. As the power is 1
minus the probability of type II error, a well-
designed study has built-in protection against mak-
ing a type II error when one of these alternatives is
true but generally does not have this protection at
other alternatives. In fact, for most statistical mod-
els used in practice, for alternatives close to the null
hypothesis, the probability of type II error is near
1 a, whereais the significance level of the test.
As the alternative hypothesis is usually composite,
not all alternatives can be protected uniformly.
Thus, accepting the null hypothesis when the test
fails to achieve statistical significance is a decision
associated with uncontrolled probability of type II
error. For this reason, statisticians prefer to declare
the test as inconclusive when it fails to achieve
statistical significance.
Confidence intervals: precision
and confidence
Testing statistical hypotheses is a decision-making
tool. The outcome of the test is a dichotomy;
either the test is declared ‘statistically significant’
or it is not. The test provides directly very little
information on the magnitude of the effect of inter-
est. In the example of the heart rate data of
Table 25.4, we have declared the test statistically
significant and rejected the null hypothesis that the
effect is zero. But we have not identified how large
the effect is. It isoften important totakethenext step
and estimate the magnitude of the effect. The
obvious starting point is the ‘signal’D¼XB
XA¼ 5 :2. This value isanestimate of the difference
between the two population means,¼BA,
and as we have already seen, it is associated with a
certain amount of variability measured by its stan-
dard error. This means that if the experiment was to
be repeated under exactly the same conditions, it is
mostlikelythatavaluedifferentthan5.2isobtained.
But how different? How much should one expect the
values obtained from repetitions of the experiment
to spread about the true? This information is
provided by the standard error.
A method of simultaneously providing informa-
tion on the magnitude of the estimated parameter
and the range of likely values of the estimate is the
confidence interval. The key idea rests on a funda-
mental mathematical fact thatifXnisa samplemean
of a variable calculated fromnindependent samples
of a variable, whose population mean and standard
error aremands, respectively, then the quantityZ¼Xn
s=ffiffiffi
np ð 2 Þhas approximately Standard Normal distribution
(Gaussian distribution). The Normal distribution
has the familiar bell-shaped curve and is tabulated
in almost any elementary statistics textbook. The
word ‘approximately’ here means that the actual
distribution ofZmay be different from the Normal
distribution, but it becomes closer and closer to it as
the sample sizenincreases. For all practical pur-
poses, when the sample size is greater than 30,
performing probability calculations onZusing
the Standard Normal Distribution tables, will result
in only minor errors.
Using the Standard Normal Distribution tables,
one can find for every number 0<g<1, a pair of
numbersZ 1 (g) andZ 2 (g), such thatProb:fZ 1 ðgÞZZ 2 ðgÞg¼ 1 g ð 3 Þ25.9 STATISTICAL INFERENCE 329