Statistical Methods for Psychology

In Experiment 1 an investigator wishes to examine the efficacy of four different treat-

ments for post-traumatic stress disorder (PTSD) in rape victims. She has chosen to use both

male and female therapists. Our experimenter is faced with two choices. She can run a one-

way analysis on the four treatments, ignoring the sex of the therapist (SexTher) variable

entirely, or she can run a 4 3 2 factorial analysis on the four treatments and two sexes. In

this case the two-way has more power than the one-way. In the one-way design we would

ignore any differences due to SexTher and the interaction of Treatment with SexTher, and

these would go toward increasing the error term. In the two-way we would take into ac-

count differences that can be attributed to SexTher and to the interaction between Treat-

ment and SexTher, thus removing them from the error term. The error term for the two-way

would thus be smaller than for the one-way, giving us greater power.

For Experiment 2, consider the experimenter who had originally planned to use only

female therapists in her experiment. Her error term would not be inflated by differences

among SexTher and by the interaction, because neither of those exist. If she now expanded

her study to include male therapists, would increase to account for additional effects

due to the new independent variable, but the error term would remain constant because the

extra variation would be accounted for by the extra terms. Since the error term would re-

main constant, she would have no increase in power in this situation over the power she

would have had in her original study, except for an increase in n.

As a general rule, a factorial design is more powerful than a one-way design only when

the extra factors can be thought of as refining or purifying the error term. In other words,

when extra factors or variables account for variance that would normally be incorporated

into the error term, the factorial design is more powerful. Otherwise, all other things being

equal, it is not, although it still possesses the advantage of allowing you to examine the in-

teractions and simple effects.

You need to be careful about one thing, however. When you add a factor that is a ran-

dom factor (e.g., Classroom) you may well actually decrease the power of your test. As you

will see in a moment, in models with random factors the fixed factor, which may well be

the one in which you are most interested, will probably have to be tested using

as the error term instead of. This is likely to cost you a considerable amount of

power. And you can’t just pretend that the Classroom factor didn’t exist, because then you

will run into problems with the independence of errors. For a discussion of this issue, see

Judd, McClelland, and Culhane (1995).

There is one additional consideration in terms of power that we need to discuss.

McClelland and Judd (1993) have shown that power can be increased substantially using

what they call “optimal” designs. These are designs in which sample sizes are apportioned

to the cells unequally to maximize power. McClelland has argued that we often use more

levels of the independent variables than we need, and we frequently assign equal numbers

of participants to each cell when in fact we would be better off with fewer (or no) participants

in some cells (especially the central levels of ordinal independent variables). For example,

imagine two independent variables that can take on up to five levels, denoted as

for Factor A, and for Factor B. McClelland and

Judd (1993) show that a 5 3 5 design using all five levels of each variable is only 25% as effi-

cient as a design using only , and. A 3 3 3 design using ,

and is 44% as efficient. I recommend a close reading of their paper.

13.8 Expected Mean Squares and Alternative Designs

For traditional experimental research in psychology, fixed models with crossed inde-

pendent variables have long been the dominant approach and will most likely continue

to be. In such designs the experimenter chooses a few fixed levels of each independent

B 1 ,B 3 , and B 5

A 1 and A 5 B 1 and B 5 A 1 ,A 3 , and A 5

A 1 ,A 2 ,A 3 ,A 4 , and A 5 B 1 ,B 2 ,B 3 ,B 4 , and B 5

MSerror

MSinteraction

SStotal

430 Chapter 13 Factorial Analysis of Variance