Statistical Methods for Psychology

the tests we have discussed. Partly to overcome this objection, Scheffé proposed that peo-

ple may prefer to run his test at a5.10. Although both SPSS and SAS only offer this test

as a pairwise test of means, it should never be used that way. There are much more power-

ful alternatives if you only want pairwise tests. Scheffé stated that when the “onlycontrasts

of interest are the^1 ⁄ 2 k(k 2 1 ) differences mi2mj, the method of Tukey... should be used

in preference to the above, because the confidence intervals will then be shorter.” It is curi-

ous that two of the major statistical packages only offer the Scheffé test in the situation for

which the test’s originator specifically recommended against its use. In general, the Scheffé

test should never be used to make a set of solely pairwise comparisons, nor should it nor-

mally be used for comparisons that were thought of a priori. The test was specifically de-

signed as a post hoc test, and its use on a limited set of comparisons that were planned

before the data were collected would generally be foolish. Unless you have a very very

large number of planned contrasts, a Bonferroni correction (modified or not) on the results

of your a priori contrasts would be more powerful. Although most discussions of multiple-

comparison procedures include the Scheffé, and many people recommend it, perhaps out

of habit, it is not often seen as the test of choice in research reports because of its conserva-

tive nature. I can’t imagine when I would ever use it, but I have to include it here because it

is such a standard test.

Dunnett’s Test for Comparing All Treatments with a Control

In some experiments the important comparisons are between one control treatment and each

of several experimental treatments. In this case, the most appropriate test is Dunnett’s test.

This is more powerful (in this situation) than are any of the other tests we have discussed

that seek to hold the familywise error rate at or below a.

We will let represent the critical value of a modified tstatistic. This statistic is found

in tables supplied by Dunnett (1955, 1964) and reproduced in Appendix. We can either

run a standard t test between the appropriate means (using as the variance estimate

and evaluating the t against the tables of ) or solve for a critical difference between

means. For a difference between means and (where represents the mean of the

control group) to be significant, the difference must exceed

Applying this test to our data, letting group S-S from Table 12.1 be the control group,

We enter Appendix with k 5 5 means and 5 35. The resulting value of is 2.56.

Thus, whenever the difference between the control group mean (group S-S) and one of the

other group means exceeds 7.24, that difference will be significant. The k 2 1 statements

we will make concerning this difference will have an FWof a5.05.

S-S versus Mc-M= 11229 =- 18

S-S versus S-M= 11224 =- 13

S-S versus M-M= 11210 = 1

S-S versus M-S= 1124 = 7

6

Critical value (Xc 2 Xj)=2.56

B

2(32.00)

8

=2.56(2.828)=7.24

td dferror td

Critical value (Xc 2 Xj)=td

B

2(32.00)

8

Critical value (Xc 2 Xj)=td

B

2 MSerror

n

Xc Xj Xc

td

MSerror

td

td

12.6 Post Hoc Comparisons 395

Dunnett’s test