Interpretation of Statistical AnalysesThe final stage in the research process is when the researcher interprets the statistical tests
and draws conclusions about circumstances in the parent population based on sample data
(step 4). You may not have consciously noticed but the four steps shown in Figure 4.1 are
interdependent, and our interpretation is therefore based on what we know about each
stage of the study. For example, the kinds of questions addressed will influence the nature
of the variables; how they are operationalized (observed/measured). This in turn will
influence, amongst other considerations, the size of the sample and how it is chosen.
Sample size and level of measurement of variables are two important characteristics that
influence the choice of a statistical test.
Interpretation of statistical results should also be related to limitations of the study
design, for example, how confident are you that randomness was built into the design?
Were there any sources of hidden systematic error (bias), for example, only certain ability
groups of pupils were involved in a study? Was an experiment realistic—a referential
communication task may be based on role play—is this something that 5- and 6-year-old
children do very often? Are there any leading questions in a survey? You should of
course relate your results to findings from similar studies.
As you may have guessed, the research process illustrated in Figure 4.1 is
oversimplified. Important questions have been conveniently overlooked, examples
include: How do you know when to reject a hypothesis? What happens if you reject a
hypothesis when you should not? What is the chance of detecting a relationship or
difference if one really exists? What is the chance of detecting a difference which does
not really exist? How large should my sample be? Is variability in my sample important?
How precise is my estimate and what confidence should I have in it? What statistical test
should I use and why? Answers to these questions affect the confidence and
trustworthiness that we have in any conclusions we draw.
I am sure you would like answers to these questions. We are, however, not quite at the
point yet where such answers would be meaningful. These questions and similar issues
are discussed in Chapter 5, ‘Choosing a statistical test’.
Finally, it once again needs to be stressed that a statistical analysis plan should not be
left until after data has been collected. The idea of statistical randomness should not just
enter a study in a haphazard way, it should be deliberately planned into the research
design if you intend to use inferential statistical tests. In the next section, the idea of
probability and its role in the research process is explored in greater depth.
4.2 Statistical ProbabilityStatistical probability forms the basis of all tests of statistical significance. Probability is a
way of assigning a number to the likelihood of the occurrence of an event or outcome.
Put another way, probability is a way of measuring chance and allows us to place the
likelihood of an outcome on a continuum ranging from certainty, which has a probability
value of 1, to impossible which has a probability of zero. The closer a probability is to 1
the more certain is the occurrence of the event.
Probability and inference 89