Statistical Methods for Psychology

Matched Samples and Related Problems

In discussing repeated-measures designs, we have spoken in terms of repeated measure-

ments on the same subject. Although this represents the most common instance of the use

of these designs, it is not the only one. The specific fact that a subject is tested several times

really has nothing to do with the matter. Technically, what distinguishes repeated-measures

designs (or, more generally, randomized blocks designs,of which repeated-measures de-

signs are a special case) from the common factorial designs with equal ns is the fact that

for repeated-measures designs, the off-diagonal elements of do not have an expectation

of zero—that is, the treatments are correlated. Repeated use of the same subject leads to

such correlations, but so does use of matched samplesof subjects. Thus, for example, if

we formed 10 sets of three subjects each, with the subjects matched on driving experience,

and then set up an experiment in which the first subject under each treatment came from

the same matched triad, we would have correlations among treatments and would thus have

a repeated-measures design. Any other data-collection procedure leading to nonzero corre-

lations (or covariances) could also be treated as a repeated-measures design.

14.12 Mixed Models for Repeated-Measures Designs

Earlier in the chapter I said that the standard repeated-measures analysis of variance

requires an assumption about the variance–covariance matrix known as sphericity, a spe-

cific form of which is known as compound symmetry. When we discussed and we were

concerned with correction factors that we could apply to the degrees of freedom to circum-

vent some of the problems associated with a failure of the sphericity assumption.

There is a considerable literature on repeated-measures analyses and their robustness

in the face of violations of the underlying assumptions. Although there is not universal

agreement that the adjustments proposed by Greenhouse and Geisser and by Huynh and

Feldt are successful, the adjustments work reasonably well as long as we focus on overall

main or interaction effects, or as long as we use only data that relate to specific simple

effects (rather than using overall error terms). Where we encounter serious trouble is when

we try to run individual contrasts or simple effects analyses using pooled error terms. Boik

(1981) has shown that in these cases the repeated-measures analysis is remarkably sensi-

tive to violations of the sphericity assumption unless we adopt separate error terms for each

contrast, as I did for the simple effects tests in Table 14.13. However there is another way

of dealing with assumptions about the covariance matrix, and that is to not make such

assumptions. But to do that we need to take a different approach to the analysis itself.

Standard repeated measures analysis of variance has two problems that we have lived

with for many years and will probably continue to live with. It assumes both compound

symmetry (or sphericity) and complete data. If a participant does not appear for a follow-

up session, even if he appears for all of the others, he must be eliminated from the analy-

sis. There is an alternative approach to the analysis of repeated measures designs that

does not hinge on either sphericity assumptions or complete data. This analysis is often

referred to as mixed models, multilevel modeling,or hierarchical modeling.There is a

bit of confusion here because we have already used the phrase “mixed models” to refer

to any experimental design that involves both fixed and random factors. That is a per-

fectly legitimate usage. But when we are speaking of a method of analysis, such as we

are here, the phrase “mixed models” refers more to a particular type of solution, in-

volving both fixed and random factors, using a different approach to the arithmetic.

More specifically, when someone claims to have done their analysis using mixed mod-

els, they are referring to a solution that employs maximum likelihoodor, more likely,

́N ~ ́

g

Section 14.12 Mixed Models for Repeated-Measures Designs 499

randomized
blocks designs

matched
samples

mixed models

multilevel
modeling

hierarchical
modeling

maximum
likelihood