Statistical Analysis for Education and Psychology Researchers

between two means also has a sampling distribution which has a mean and a standard
deviation, the latter is referred to as the standard error of the difference in means.
The sensitivity of the t-test in detecting differences is dependent upon the total
variability in scores (standard error of the difference in means). If the overall variability
in scores is minimal then only a small difference between means of the two groups might
reflect a consistent and significant difference. However, if there is large overall
variability in scores then a larger difference between means is required to attain statistical
significance. It is more difficult to detect real differences with heterogeneous groups
because more of the variability in individuals’ scores may be due to error or other
(unmeasured) effects rather than the predicted differences of interest. The implication for
research design is that you are more likely to detect a significant difference between
groups if overall scores are homogeneous. See also the discussion of power analysis in
Chapter 5.
When samples are small (n<30) the sample standard deviation may not be a good
estimator of the unknown population standard deviation (with small samples, the sample
standard deviation underestimates the population standard deviation more than half the
time) and consequently the ratio of the difference between means to the standard error of

the difference diff in means, may not have a normal distribution. This
ratio is called a t-statistic and when the variances in both samples are similar, the t-
statistic has a probability distribution known as the Student’s t-distribution.
The shape of the t-distribution changes with sample size, that is there is a different t-
distribution for each sample size, so when we use the t-distribution we also need to refer
to the appropriate degrees of freedom which is based on the sample size and the number
of parameters estimated. As the sample size increases above 30, the t-distribution
approaches a normal distribution in shape.

Statistical Inference and Null Hypothesis

We use the t-test to see whether there is a difference between two means, the null
hypothesis is therefore, H 0 :μ 1 −μ 2 =0; this is equivalent to μ 1 =μ 2. In words, this says that
the population means are the same, which is equivalent to saying there is one population
and not two. The alternative hypothesis is either nondirectional, H 1 :μ 1 ≠μ 2 , rejection
region |t|>t 1 −α/2 (this means that the absolute value of t is greater than the critical value of t
at the 0.025 level of significance, if alpha is 5 per cent) or it may be one-sided, μ 1 >μ 2 or
μ 1 <μ 2 , rejection region t>t 1 −α or t<−t 1 −α. The sampling distribution of the difference
between means is used to test this null hypothesis.

Pooled Variance Estimate of t-ratio (equal variance)

The t-statistic has an exact distribution only when the two populations have the same
variance. This is called the homogeneity of variance assumption. When a pooled
estimate of the population variance, σ^2 , is used in the calculation of the t-ratio it is
referred to as a pooled variance estimate. This is given by the formula:

Pooledvariance

estimate—8.15

Inferences involving continuous data 293