Statistical Analysis for Education and Psychology Researchers

freedom which are associated with smaller critical test statistic values for attaining a
specified level of statistical significance. For example, a critical t-value with 10 df, α two-
tailed test and p≤0.05 is 2.228, and if df increases to 25, all other attributes remaining the
same, then the critical t-value is only 2.060.

Variability of population measures and statistical power

The more homogeneous groups are (less variability), the easier it is to detect differences
(relationships). Even when random samples are used, if the measure of interest is
heterogeneous with respect to the population, then real treatment effects will be more
difficult to detect than would be the case if measures were homogeneous.
The researcher has direct control over the sample size and can therefore increase the
statistical power of a design by increasing the sample size. A researcher can do little
directly about the population variability, denoted by sigma squared σ^2 , and hence the

standard error, When population variability is large statistical power is reduced.
You should note that by increasing the sample size this has the effect of reducing the
standard error and hence increasing statistical power.

Alpha Type I error rate and statistical power

Generally in experimental and survey designs we try to minimize α, that is the problem of
finding a difference that does not actually exist in the population. However, the alpha
Type I error rate is inversely related to beta Type II error rate, the problem of not finding
a difference that does exist in the population. As we increase alpha, at the same time we
reduce beta and hence increase statistical power (1−β). The larger the chosen α or
significance level, for example, p≤0.10 rather than the conventional p≤0.05, then the
smaller is the critical t-ratio required for statistical significance and hence the easier it is
to attain significant difference. Also, the direction of any differences tested, such as a
one-tailed or a two-tailed test influences the attainment of statistical significance. For a
chosen alpha, a one-tailed test (one-direction test, such as H 1 : either μ 1 >μ 2 or μ 1 <μ 2 ) will
be significant at a smaller critical t-ratio value than a comparable two-sided test.

Effect size and statistical power

The effect size for a given difference between sample means can be defined as the ratio
of the size of difference between sample means divided by the population standard
deviation. In notational form this is:

Effect

size—

5.1

When calculating an effect size from sample data sample means replace μ 1 and μ 2 and the
pooled standard deviation replaces the population standard deviation. The pooled
standard deviation for two samples is evaluated as:

Choosing a statistical test 133