Statistical Analysis for Education and Psychology Researchers

Choice of statistical tests

After data collection, descriptive statistics are used to summarize data. The research
questions addressed and the nature of the data will influence the choice of summary
statistics. For example, if more scores are at one extreme rather than in the middle of a
distribution, the median may be a more appropriate measure of central tendency than the
average. If differences between mean scores are to be estimated then the laws of chance
may be used to say whether differences among the means of treatment groups are
statistically significant and to indicate whether differences are too large to be
attributable to chance alone. Thought should also have been given to appropriate analyses
at the planning stage. A common significance test for comparison of two independent
groups is the independent t-test, which makes certain assumptions about how data is
generated: the sampling, how the data is measured and the variability of the data (these
considerations are discussed in Chapter 8). In other studies, for example, in the evaluation
of a maths programme it may be more appropriate to measure individual change, that is
the difference between before and after scores. The important statistic in this example
would be the mean of the difference scores (after ‘minus’ before) rather than the
difference between means of independent groups.

Being a critical research consumer

Most empirical researchers will be consumers of research reports and papers. It is
necessary to be able to discern good studies from poor ones, to be able to identify
limitations in design and interpretation and hence judge the dependability of conclusions.
As we will see in later chapters, not all studies published in educational research journals
meet the statistical standards that we might expect, therefore publication should not be
seen as a guarantee of quality and trustworthiness. The novice researcher may find this
difficult to believe but as researchers become more experienced and knowledgable they
also become more critical and rely more on their own judgments than on the judgments
of others.
Whenever we encounter research findings we need to consider their trustworthiness. A
particular sample statistic such as the sample average may be calculated and used to
estimate the mean for the population from which the sample was drawn. As one
particular sample average is likely to vary from a second independent sample average,
even if exactly the same procedure for selection and calculation is used (all sampling
results are subject to sampling errors), then the laws of chance can be used to provide an
estimate of precision or a confidence interval for the obtained sample average. When
reading a research report we should consider the confidence intervals around any sample
statistics given. It is good practice to report the confidence interval or to give sufficient
information for the confidence interval to be calculated (see Chapters 6 and 8).
Survey research is seldom confined to drawing conclusions only about the subjects in
an achieved sample. Even if this is intended, often implicit generalizations will be made,
if not by the researcher then by the research consumer, to the parent population, or to
similar subjects. In this case the trustworthiness of the results will depend not only upon
the precision of the sample statistic but also on the extent to which the picture given by
the sample represents the situation in the population. Clearly the survey design, that is
how the data were measured, collected and recorded, will influence estimates of precision

Statistics and research design 5