Statistical Analysis for Education and Psychology Researchers

For example, a research student wanted to assess the effectiveness of vocabulary
acquisition among primary school pupils under three teaching approaches (treatments):
storytelling with pictures, storytelling alone and silent reading. Thirty target word
meanings of equal difficulty were selected (factors considered included context, word
length, word type classification), and ten words were used in each of the three treatment
conditions. For each treatment condition, if a subject scored better than chance on the ten
target words the subject was assigned a score of 1, success. If the subject did not score
better than chance, the treatment response was assigned a score of 0. Data for this design
can be arranged in a contingency table where columns represent treatments or
measurement occasions and subjects form the rows of the table, for example:
Treatments (measurement occasions)
Subjects Storytelling +pictures Storytelling alone Silent reading Row totals
1 1 0 1
2 1 0 0
3 0 1 0
4 1 0 0

..
..
..
Column Totals Grand total N

The purpose of the test is to determine whether related sets of binary observations differ.
When the related measures correspond to a series of observations on the same subjects
under different treatment conditions the Q statistic provides a test of whether the
treatments differ significantly.

Statistical Inference and Null Hypothesis

The null hypothesis is that the population parameter for the probability of a success under
each treatment condition is the same. That is the treatments are equally effective and any
variation in the column totals is simply due to sampling variation. The alternative
hypothesis is that the treatments do not have the same effect.

Test Assumptions

Cochran’s Q test may be used when the following assumptions are met:

1 The response variable is binary and observations are related, same subjects observed
under different treatment conditions (measurement occasions), or matched subjects.
2 The sampling distribution of the Q statistic approximates to a χ^2 distribution with
degrees of freedom as the number of columns −1. Under the null hypothesis the
probability associated with the value of Q as large as the observed value is determined
by reference to the χ^2 distribution. However, this approximation is only valid when the
product of the number of subjects (rows) and treatments (columns) is ≥24. When the
product of rows and columns is <24 tables of exact distributions should be consulted,
see Patil (1975).

Statistical analysis for education and psychology researchers 196