Statistical Methods for Psychology

variable, which are the levels that are of primary interest and would be the same levels

he or she would expect to use in a replication. In a factorial design each level of each

independent variable is paired (crossed) with each level of all other independent

variables.

However, there are many situations in psychology and education where this traditional

design is not appropriate, just as there are a few cases in traditional experimental work. In

many situations the levels of one or more independent variables are sampled at random

(e.g., we might sample 10 classrooms in a given school and treat Classroom as a factor),

giving us a random factor.In other situations one independent variable is nested within

another independent variable. An example of the latter is when we sample 10 classrooms

from school district A and another 10 classrooms from school district B. In this situation

the District A classrooms will not be found in District B and vice versa, and we call this a

nested design.Random factors and nested designs often go together, which is why they

are discussed together here, though they do not have to.

When we have randomand/or nested designs, the usual analyses of variance that we

have been discussing are not appropriate without some modification. The primary problem

is that the error terms that we usually think of are not correct for one or more of the Fs that

we want to compute. In this section I will work through four possible designs, starting with

the traditional fixed model with crossed factors and ending with a random model with

nested factors. I certainly can not cover all aspects of all possible designs, but the general-

ization from what I discuss to other designs should be reasonably apparent. I am doing this

for two different reasons. In the first place, modified traditional analyses of variance, as de-

scribed below, are quite appropriate in many of these situations. In addition, there has been

a general trend toward incorporating what are called hierarchical modelsor mixed models

in our analyses, and an understanding of those models hinges crucially on the concepts dis-

cussed here.

In each of the following sections, I will work with the same set of data but with different

assumptions about how those data were collected, and with different names for the inde-

pendent variables. The data that I will use are the same data that we saw in Table 13.2 on

Eysenck’s study of age and recall under conditions of varying levels of processing of the

material.

One important thing to keep firmly in mind is that virtually all statistical tests operate

within the idea of the results of an infinite number of replications of the experiment. Thus

the Fs that we have for the two main effects and the interaction address the question of “If

the null hypothesis were true and we replicated this experiment 10,000 times, how often

would we obtain an Fstatistic as extreme as the one we obtained in this specific study?” If

that probability is small, we reject the null hypothesis. There is nothing new there. But we

need to think for a moment about what would produce different Fvalues in our 10,000

replications of the same basic study. Given the design that Eysenck used, every time we re-

peated the study we would use one group of older subjects and one group of younger sub-

jects. There is no variability in that independent variable. Similarly, every time we repeat

the study we will have the same five recall conditions (Counting, Rhyming, Adjective, Im-

agery, Intention). So again there is no variability in that independent variable. This is why

we refer to this experiment as a fixed effect design—the levels of the independent variable

are fixed and will be the same from one replication to another. The only reason why we

would obtain different Fvalues from one replication to another is sampling error, which

comes from the fact that each replication uses different subjects. (You will shortly see that

this conclusion does not apply with random factors.)

To review the basic structural model behind the analyses that we have been running up

to now, recall that the model was

Xijk=m1ai1bj1abij 1 eijk

Section 13.8 Expected Mean Squares and Alternative Designs 431

crossed

random factor

nested design

random designs

hierarchical
models

mixed models