Statistical Methods for Psychology

when our interest is in a specific subset of contrasts. In general, however, multistage

procedures would not be used as a substitute when making all pairwise comparisons

among a set of means.

As you saw, the Bonferroni test is based on the principle of dividing up FWfor a fam-

ily of contrasts among each of the individual contrasts. Thus, if we want FWto be .05 and

we want to test four contrasts, we test each one at a5.05 4 5 .0125. The multistage tests

follow a similar principle, the major difference being in the way they choose to partition a.

Holm and Larzelere and Mulaik Tests

Both Holm (1979) and Larzelere and Mulaik (1977) have proposed a multistage test that

adjusts the denominator (c) in 5acdepending on the number of null hypotheses

remaining to be tested. Holm’s test is generally referred to when speaking about the analy-

sis of variance, whereas the Larzelere and Mulaik test is best known as a test of signifi-

cance for a large set of correlation coefficients. The logic of the two tests is the same,

though the method of calculation is different.

In the Holm procedure, we calculate values of just as we did with the Bonferroni t

test. For the equal ncase, we compute

For the unequal ncase, or when we are concerned about heterogeneity of variance, we

compute

We calculate for all contrasts of interest and then arrange the values in increas-

ing order without regard to sign. This ordering can be represented as

, where cis the total number of contrasts to be tested.

The first significance test is carried out by evaluating , the largest of the , against the

critical value in Dunn’s table corresponding to ccontrasts. In other words, is evaluated at

. If this largest is significant, then we test the next largest against
the critical value in Dunn’s table corresponding to c 2 1 contrasts. Thus, is evaluated
at. The same procedure continues for... until the test re-
turns a nonsignificant result. At that point we stop testing. Holm has shown that such a pro-
cedure continues to keep FW#a, while offering a more powerful test.
The rationale behind the test is that when we reject the null for tc, we are declaring that
null hypothesis to be false. If it is false, that only leaves c 2 1 possibly true null hypotheses,
and so we only need to protect against c 2 1 contrasts. A similar logic applies as we carry
out additional tests. This logic makes particular sense when you know, even before the ex-
periment is conducted, that several of the null hypotheses are almost certain to be false. If
they are false, there is no point in protecting yourself from erroneously rejecting them.
To illustrate the use of Holm’s test, consider our example on morphine tolerance. With
the standard Bonferroni t test, we evaluated four contrasts with the following results,
arranged by increasing magnitude of , as in Table 12.3.
If we were using the Bonferroni test, each of these s would be evaluated against
which is actually Student’s tat a50.0125. For Holm’s test we vary the criti-
cal value in stages, depending on the number of contrasts that have not been tested. This

t¿.05=2.64,

t¿

t¿

a¿=a>(c 2 1) tc¿ 22 , tc¿ 23 , t¿c 24 ,

t¿c 21

a¿=a>c t¿ t¿(i.e.,tc¿ 21 )

t¿c

t¿c t¿

... ƒt 3 ¿ƒ ...Á... ƒt¿cƒ

ƒt 1 ¿ƒ ... ƒt¿ 2 ƒ

t¿ t¿

t¿=

Xi 2 Xj

B

s^2 i

ni

1

s^2 j

nj

t¿=

Xi 2 Xj

B

2 MSerror

n

t¿

a¿ >

>

380 Chapter 12 Multiple Comparisons Among Treatment Means