Statistical Analysis for Education and Psychology Researchers

A probability model for the values of a variable in a population is a mathematical
description or a kind of model for randomness which gives the probabilities for all
possible values of the variable.

Example 4.1: Use of Non-probability Samples

Non-probability samples are usually not representative of the population of interest and
generally, therefore, should not be used for statistical inference. In practice however they
often are. For example, it is not uncommon in experimental designs that involve school
children as subjects, to use for a sample those children that were nominated by their class
teacher. Sometimes, for administrative convenience, an entire class is used as the sample,
for example, a common programme evaluation design involves the administration of an
appropriate ‘before’ and ‘after’ test to an entire class of children who are all programme
participants.
Alternatively, teachers may not select pupils for participation in an experimental study
on the grounds of chance alone; they may have other reasons for selecting pupils which
the researcher may be unaware of, for example, a talkative child, a nuisance, or because a
teacher thinks that a particular child will ‘perform’ best.
Considering a typical ‘before’ and ‘after’ design then if all participants in the
programme were tested, they either represent the entire population of interest and
therefore there is no need to use statistical inference, or as is often the case, it is not clear
whom the population of interest is, in which case it cannot be a probability sample.

So, if in a journal article you read about a study which was similar to the

‘before’ and ‘after’ design mentioned above, how should you interpret the

reported results of the statistical analysis?

In general we are asked to assume that we are dealing with a probability sample which
is representative of an often undefined population. If this assumption is not reasonable,
perhaps you are not convinced by the evidence provided by the author(s), then the
statistical test(s) will be invalid, and there is a good chance the study conclusions will be
also.

Linking Probability and Inference

A key link between the ideas of probability and statistical inference is the sample
statistic, or more precisely, the sampling distribution of the sample statistic. The
sampling distribution of any statistic is a distribution of the values of that statistic (not
the raw scores) when separate independent random samples of equal size are drawn from
the same population. The sampling distribution of a statistic is actually a probability
distribution which describes the behaviour or likely values of a sample statistic in
repeated sampling, provided the data are produced by a random process. This explains
the importance of random sampling or randomization when collecting data. The term
probability distribution has been slipped in here without explanation. We shall defer

Probability and inference 87