Chapter 18, which should not be difficult to understand even without reading the intermediate
chapters.11.10 Fixed versus Random Models
We have not said anything about how we choose the levels of our independent variable; we
have simply spoken of “treatments.” In fact, if you think about it, we could obtain the lev-
els of the treatment variable in at least two different ways: We could, and usually do, delib-
erately select them or we could sample them at random. The way in which the levels are
derived has implications for the generalizations we might draw from our study.
Assume that we were hired as consultants by the Food and Drug Administration (FDA)
and asked to run a study to compare the four most popular pain relievers. We will have four
treatment levels (corresponding to the four pain relievers) that were selected by the FDA.
If we chose to replicatethe study (run it over again to verify our results), we would use ex-
actly the same levels (drugs). In a sense, the treatment levels actually used have exhausted
the levels of interest. The important point here is that the levels are in fact fixedin the sense
that they do not change randomly from one replication of the study to another. The analy-
sis of such an experiment is referred to as a fixed-model analysis of variance.
Now assume that we are hired by the FDA again, but this time they merely tell us to
compare a number of pain relievers to see whether “one brand is as good as the next.” In
this case, it would make sense to select randomlythe pain relievers to be compared from
the population of all available pain relievers. Here the treatment levels are the result of a
random process, and the population of interest with respect to pain relievers is quite large
(probably over 50). Moreover, if we replicated this study we would again choose the brands
randomly, and would most likely have a whole new set of brands to compare. Because of
the process by which treatment levels are obtained, we speak of treatments as a random
variable and of the analysis as a random-model analysis of variance.
We will have much more to say about fixed and random models in Chapters 13 and 14.
They are playing an expanded role in the analysis of research in the behavioral sciences,
and you need to understand them. The important point at this time is that in a fixed model,
the treatment levels are deliberately selected and would remain constant from one replica-
tion to another. In our example of a fixed model, we actually set out to compare, for exam-
ple, Bayer Aspirin with Tylenol. In a random model, treatment levels are obtained by a
random process and would be expected to vary across replications. In our example of a ran-
dom model, we were studying pain relievers, and the ones that we happened to use were
just random samples of pain relievers in general. For a one-way analysis of variance, the
distinction is not particularly critical, but it can become quite important when we use more
complex designs where we not only have to deal with random variables, but often with
what are called “nested variables” as well. In more complex models the independent vari-
able that is random is often not of great importance in its own right. It is often there prima-
rily to increase the generalizability of our study. However, its presence can substantially
affect the resulting Fvalues.11.11 The Size of an Experimental Effect
The fact that an analysis of variance has produced a significant Fsimply tells us that there
are differences among the means of treatments that cannot be attributed to error. It says
nothing about whether these differences are of any practical importance. For this reason, we
must look beyond the value of Fto define an additional measure reflecting the “importance”Section 11.11 The Size of an Experimental Effect 343replicate
fixed-model
analysis of
variance
random-model
analysis of
variance