Statistical Analysis for Education and Psychology Researchers

under more than one treatment combination, this is referred to as a repeated measures
design.

Example 3

Involvement of friends and relations in the completion of a hearing handicap inventory
(HHI) prior to rehabilitative audiological assessment and hearing aid fitting is believed to
increase clients motivation to use a hearing aid. A research student wanted to know
whether clients who used the HHI were on average well motivated to use their hearing
aid. The researcher chose to investigate this question for her thesis study. She decided to
do a survey to study the impact of the pre-assessment HHI on clients’ motivation to use
their hearing aids.
The first 15 clients who were scheduled to attend a hearing aid centre were sent a brief
explanatory letter and the HHI with their appointment card. They were asked to bring
along the completed inventory to their appointment. The next 25 clients

scheduled to attend the clinic received a brief explanatory letter inviting them to
participate in the study to form a comparison group. They did not receive the HHI.
At the clinic assessment, clients in both groups completed a motivation questionnaire
relating to their use of hearing aids. A motivation score was computed for each client.
The researcher expected there to be a statistically significant difference in mean
motivation score in favour of the HHI group. After data had been collected the researcher
sought advice on which statistical test to use.

The study in Example 3 is an experiment and not, as the researcher thought or intended a
survey. It is an experiment because the clients, the experimental subjects, had something
done to them. The intention was to observe a response and attribute cause and effect
relations. In a survey the subjects in a sense are passive, that is they are not exposed to
treatments. The intention is usually to estimate a population characteristic, in this
example clients’ average motivation to use their hearing aids.
The experimental design chosen did not address the original research question, and in
any case has a number of design flaws which render any subsequent analysis at best
tenuous. Some of the problems with this design include:

• no random allocation of subjects to experimental groups;


• no inclusion or exclusion criteria specified, that is control for potentially confounding
  influences, such as previous hearing aid users, stroke patients, age of clients;


• no control over the intervention, that is how the HHI was used by clients;


• not a balanced design, that is unequal numbers of subjects in the two groups;


• no mention of what would be a meaningful difference in average motivation scores, that
  is the magnitude of the expected effect.

This example illustrates the importance of considering statistical principles at the design
stage. Here thought should have been given to questions such as:

• How many individuals do we need in total for this study?


• How will individuals be selected to participate?

Statistical analysis for education and psychology researchers 14