Statistical Analysis for Education and Psychology Researchers

or more groups) are independent and interest is focused on association between different
samples (groups) then an extension of the r×2 Sample χ^2 test of association, referred to as
an r×k Sample χ^2 test, should be considered. When three or more related samples are to
be compared on, for example a true/ false or pass/fail basis, i.e., binomial data, then the
Cochran’s Q-test should be considered. It can be thought of as an extension of the two
sample proportions test, the null hypothesis being that the proportions are equal in each
group.

Research Design

Choice of an appropriate statistical test depends upon whether the samples of scores or
observations are independent or related. Generally a related measures design is preferable
to an independent samples (groups) design because statistically there are fewer degrees of
freedom. Hence statistical significance is attained, at a specified p-level, with a smaller
difference. Also a practical benefit is that fewer subjects are required and individual
differences between conditions can be eliminated or accounted for. A disadvantage is the
need to counterbalance possible order effects and the requirement for a wash-out period
between measurement occasions. The effects of measurement or participating in an
experiment may carry over to the second measurement occasion. There are some designs
where independent samples have to be used, for example, in an investigation of
differences between boys and girls in their coping skills in different social settings.
The number of samples (groups) in a design can become complex. Various
combinations are summarized below. These descriptions are referred to in Figure 5.1.
One
sample

when a single random sample of observations is obtained from a defined population.

Two
sample

this design can take two forms:

• when two independent (separate and not related) random samples of observations are
  obtained from a defined population;


• when two samples of observations are obtained, but the two observations come from the
  same individuals (related).
  Multiple samples (or multiple groups)
  when there are more than two samples of observations. These designs can be split into two
  types depending upon the number of independent variables (factors):


• when there is just one independent variable (factor) with different levels (categorical)
  forming groups and one response variable;


• when there is more than one independent variable (factor) and one response variable, i.e.,
  two independent variables with two or more levels (categorical) forming the groups. For
  example, a 2×2 design would be two independent variables each with two levels forming
  four groups.

Data Distributions

Data distributions can be classified into: i) binomial/nominal; ii) ranked; and iii)
continuous. These distributions have different underlying probability models and can be
thought of as three distinct classes of statistics. Here we group binomial and nominal as
one category of discrete measurement although they have different underlying

Choosing a statistical test 125