Statistical Analysis for Education and Psychology Researchers

a prediction study. A researcher, for example, may be interested in whether A-level
grades predict degree performance. The variable predicted, in this example degree
performance, is called the criterion or response variable. The variable(s) upon which
the prediction is made are called predictor or explanatory or independent variables.
See for example the study described by Peers (1994). The most common failings in
survey designs are i) the use of non-probability and non-representative samples of
subjects when the researcher wishes to generalize findings to a wider population and ii)
small sample sizes which provide imprecise sample statistics with large sampling errors.

Example 1

A research student was interested in investigating provision and practice for staff
development in secondary schools. This is an extract from the student’s research plan.

The overall purpose of the study will be to inform national policy on staff

development in secondary schools. As part of the study a structured

interview

schedule will be designed to determine what principals think of school-

based staff development, the importance they attach to it and their

willingness to involve themselves in staff development. Information will

also be collected on respondents’ age, sex, years of experience in post and

academic qualifications. A sample of 15 principals, personally known to

the researcher, in one of four geographical regions will be selected for

inclusion in the study.

Given the purpose of the study, this was not an appropriate design. A probability-based
sampling strategy would have been preferable, possibly using region as a stratification
factor. This involves taking a random sample from each of the four regions (the
stratification factor). The number of interviews would need to be increased if the
sampling error were to be kept to a reasonable level. The sampling design chosen, a non-
probability convenience sample, may also introduce bias into the study. It is possible that
colleagues would give favourable responses. Any generalizations beyond this
convenience sample would not be reasonable and therefore the main purpose of the study
would not be addressed.

This example illustrates the importance of matching design and methods to research
purpose. It also demonstrates the value of planning prior to data collection. Given the
problems identified with this design a number of alternatives exist:

• Use a probability sampling strategy and increase sample size, that is the number of
  interviews, to obtain reasonable precision on the most important variables.


• Use the convenience sample but change the aims of the study from descriptive to
  explanatory. For example, research questions might include exploration of

Statistics and research design 9