Statistical Methods for Psychology

level of significance, rather than always using at the alevel of significance. This suggestion

was then modified by Einot and Gabriel (1975) to set

and then again by Welsch (1977) to keep the Einot and Gabriel suggestion but to allow

to remain at afor r 5 k, and r 5 k 2 1. These changes hold the overall familywise error

rate at awhile giving greater power than does Tukey to some comparisons. (Notice the

similarity in the first two of these suggestions to the way ais adjusted by the Bonferroni

and the Dunn- Sidák procedures.)ˇ

What these proposals really do is to allow you to continue to use the tables of the Stu-

dentized Range Distribution, but instead of always looking for at a5.05, for example,

you look for at a5 , which is likely to be some unusual fractional value. The problem

is that you don’t have tables that give at any values other than a5.05 or a5.01. Com-

puter software can compute the necessary values without blinking an eye, and, since

almost all computations are done using software, there is no particular problem.

One way that you can run the Ryan procedure(or the Ryan/Einot/Gabriel/Welsch pro-

cedure) is to use SPSS or SAS and request multiple comparisons using the REGWQ

method. The initials refer to the authors and to the fact that it uses the Studentized Range

Distribution (q). For those who have access to SPSS or other software that will implement

this procedure, I recommend it over either the Newman–Keuls or the Tukey, because it ap-

pears to be the most powerful test generally available that still keeps the familywise error

rate at a. Those who don’t have access to the necessary software will have to fall back on

one of the more traditional tests. The SAS output for the REGWQ procedure (along with

the Student–Newman–Keuls, the Tukey, and the Scheffé tests) are presented later in the

chapter so that you can examine the results. In this situation the conclusions to be drawn

from the REGWQ and Tukey tests are the same, although you can see the difference in

their critical ranges.

The Scheffé Test

The post hoc tests we have considered all involve pairwise comparisons of means. One of

the best-known tests, which is both broader and more conservative, was developed by

Henry Scheffé (1953). Scheffé was impressed by Tukey’s concept of a family error rate

and set out to create a test that would allow any kind of contrast (pairwise or not, a priori

or post hoc) and would hold the familywise error rate at for the entire set. The Scheffé

test,which uses the Fdistribution rather than the Studentized range statistic, sets the fam-

ilywise error rate at aagainst all possible linear contrasts, not just pairwise contrasts. If

we let

then

Scheffé has shown that if is evaluated against —rather

than against —the FWis at most a. (Note that all that we have done is to cal-

culate Fon a standard linear contrast, but we have evaluated that Fagainst a modified crit-

ical value.) Although this test has the advantage of holding constant FWfor all possible

linear contrasts—not just pairwise ones—it pays a heavy price; it has the least power of all

Fa(1, dferror)

Fobt (k 2 1)Fa(k 2 1, dferror)

F=

nc^2

aa

2

jMSerror

c= aajXj and SScontrast=

nc^2

ga^2 j

a

qr

qr ar

qr

ar

ar= 12 (12a)^1 >(k>r)= 12 (12a)r>k

qr

394 Chapter 12 Multiple Comparisons Among Treatment Means

Ryan procedure

REGWQ

Scheffé test