Statistical Analysis for Education and Psychology Researchers

Test Assumptions

The Chi-square test is widely used but is also one of the most misused statistical
procedures. Basic assumptions of both χ^2 tests of independence and homogeneity for 2×2
tables are:

1 Observations are representative of the populations of interest.
2 Data is in the form of observed frequency counts.
3 Observations should be independent, that is, the probability of an observation falling in
any particular row of a contingency table does not depend on which column it is in
(and vice versa).
4 Observations should fall in only one cell of a contingency table.
5 The χ^2 test should not be used when any expected cell frequencies (see later for
computational procedure) are small because the probability distribution of χ^2 gives a
poor approximation to the sampling distribution of the χ^2 statistic. There has been
considerable debate in the literature about what constitutes small expected frequencies.
In a seminal paper on the use and misuse of the Chi-square test, Lewis and Burke
(1949) claimed that small expected frequencies were the most common weakness in
the use of Chi-square tests (p. 460). They suggested expected values of 5 as the
absolute lowest limit. More recently Camilli and Hopkins (1978) suggest that provided
the total sample size is ≥20, then expected frequencies in one or two cells can be as
low as 1 or 2. Delucchi (1983), in reviewing the literature, concluded that the Chi-
square test was a robust procedure and expected cell frequencies of <5 did not
substantively effect the Type I error rate. The general view would seem to be that
small expected frequencies are acceptable in at least one or two cells provided the
overall sample size ≥20.
Many statistical texts suggest using Yate’s (1934) correction for continuity (add
0.5 to observed cell frequencies) with small sample sizes in 2×2 tables. The
variables in a contingency table are discrete but χ^2 is a continuous distribution,
therefore adding 0.5 to each observed cell frequency is believed to improve the
Chi-square approximation. Use of this correction is also contested in the statistical
literature (not on theoretical grounds but based on its application). On balance it is
suggested that Yate’s correction should not be used because it results in
unnecessary loss of power and conservative probability estimates. With small
sample sizes, Fisher’s exact test (Fisher, 1935) should be used (see section 6.3).

Examples from the Literature

One sample χ^2 test of independence

Cope et al. (1992) invited students who had completed a part-time in-service MEd
programme to complete a questionnaire relating to their experience of the course and the
impact it was perceived to have had on them. One of the questions asked related to the
professional significance of the programme. The investigators commented that one of the
most frequently reported categories of effect was an increased understanding of
educational issues. Data as presented in the original paper is shown in Figure 6.1.

Inferences involving binomial and nominal count data 165