Statistical Analysis for Education and Psychology Researchers

Diagrammatically this is shown in Figure 4.4. The significance level (p-value) chosen by
a researcher equals the probability of making a Type I error, that is when the results of a
statistical test lead to rejection of the null hypothesis given that the null hypothesis is true.
The selected probability (or alpha level) is in fact a conditional probability. For
discussion of this point see Dracup (1995).
Population Circumstances
Conclusion from
statistical test:

Population means differ, i.e.,

H 0 is not true

Population means do not differ,

i.e., H 0 is true

Significant difference

(REJECT H 0 )

Correct decision Power p=1−βType I Error Researcher sets

p=α (alpha)

Figure 4.4: Statistical power and Type

I error

There are two notable features about Figure 4.4. The idea of statistical power is
introduced and alpha (α) is defined as the probability of occurrence of a Type I error. The
probability of a Type I Error, a, is set by the researcher (usually at p ≤ 0.05). It is under
the direct control of the researcher. The power of a statistical test is the probability of
detecting a true difference should one exist, or put another way the probability of
rejecting a null hypothesis given that it is false. The statistical power is not set directly by
the researcher rather it is related to the chosen α. As alpha increases so the power
decreases and vice versa.

Type II Error

Another way a wrong decision could be made is to fail to reject a null hypothesis that is
in fact not true. Suppose in reality, μt does not equal μc, that is the population means do
differ. In this case we could make a Type II Error if we do not reject H 0. The probability
of a Type II error is denoted by beta (β). This is shown in Figure 4.5:
Population Circumstances
Conclusion from
statistical test:

Population means differ, i.e.,

H 0 is not true

Population means do not differ,

i.e., H 0 is true

No significant

difference

Type II Error (Do not reject

H 0 )

p=β

Correct decision

p=1−α

Figure 4.5: Type II error

The decision-making process following a statistical test of the null hypothesis is
summarized in Figure 4.6:

Probability and inference 113