Statistical Analysis for Education and Psychology Researchers

a coin the proportion or long-term relative frequency of heads will approach 0.5, that is
half of the outcomes will be heads and half will be tails.
Clearly not all situations of interest would lend themselves to this long-run
interpretation. An individual usually only takes a test once, not repeatedly under the same
conditions, and we could never observe an infinite number of flips of a coin. The best we
could do would be to observe a very large number of flips of the coin. In these kinds of
circumstances we make use of a theoretical probability distribution. That is a
mathematically determined probability distribution, which describes the relative
frequency of outcomes in an infinite number of experiments, each possible outcome
having a probability value on the scale 0 to 1. To be able to use the idea of probability
and these expected long-term patterns, research designs have to have randomness or
chance planned into them. For example, random sampling or randomization in
experimental designs. Randomization is not designed into a study just to prevent bias, it
is essential if statistical inference is used, that is, whenever we collect a sample of data,
test a hypothesis using a test of statistical significance, and derive conclusions about a
population of interest. Statistical inference relies on the laws of probability which in turn
assume observations or random variables result from a random process.

4.3 Sampling Distributions

Probability theory allows us to describe mathematically the outcome of random events.
One aspect of interest to statisticians is the behaviour of sample statistics and test
statistics in the long-run. Provided data is generated by a random process, the values of
sample statistics are random and we can use probability theory to describe how a statistic
will vary with repeated samples of the same size from the same population. This idea of
repeated sampling leads to the concept of the sampling distribution of a statistic. We can
think of the sampling distribution of a sample statistic as the distribution of the values
of that statistic over repeated samples of the same size from the same population.
Although individual values of a variable may differ and individual sample statistics
repeatedly sampled from the same population differ, it is perhaps by now no surprise that
the sampling distribution of a sample statistic has a predictable and regular pattern.
Statistical inference depends upon the predictable pattern of the sampling distribution.

Sampling Distribution of Test Statistics

Whereas each descriptive statistic, mean, proportion, median variance, etc., has its own
sampling distribution, the shape of these distributions differ. Another group of statistics
called test statistics also have unique sampling distributions. Test statistics such as t, F,
and χ^2 , are all associated with specific statistical tests and similar to the descriptive
statistics, each has its own computational formulas and sampling distribution. Statistical
tables shown in the appendix of many statistical texts are simply tables of expected
outcomes or probabilities based on theoretical sampling distributions of descriptive
statistics and test statistics.

Probability and inference 91