Statistical Methods for Psychology

where is taken from the tables of the Studentized range statistic on

degrees of freedom. This is basically the solution referred to earlier in the discussion of

multiple t tests, although here we are using the Studentized range statistic instead of t, and

is an optional solution in SPSS. This solution is laborious, but the effort involved is still

small compared to that of designing the study and collecting the data. The need for special

procedures arises from the fact that the analysis of variance and its attendant contrasts are

especially vulnerable to violations of the assumption of homogeneity of variance when the

sample sizes are unequal. Moreover, regardless of the sample sizes, if the sample variances

are nearly equal you may replace and in the formula for with from the

overall analysis of variance. And regardless of the sample size, if the variances are hetero-

geneous you should probably use the Games and Howell procedure.

The Newman–Keuls Test

The Newman–Keuls testis a controversial test. I covered this procedure in the first five

editions, but have finally given in to those who argue with its underlying logic. All I will

say here is that the Newman-Keuls, often called the Student-Newman-Keuls, does not test

all comparisons as if r 5 5, but, instead, continually readjusts rdepending upon the means

being compared. This allows for means that are closer in an ordered series to be tested with

a smaller critical value than means that are further apart. (As a result, the Newman-Keuls

concludes that group M-S is different from all other groups, with M-M and S-S forming a

homogeneous subset). Unfortunately, this adjustment to rand the critical value allows FW

to exceed .05, which many people find a critical flaw. We will have little to say about the

Newman–Keuls test after this, although it is produced by most statistical software.

The Ryan Procedure (REGWQ)

As we have seen, the Tukey procedure controls the familywise error rate at aregardless of

the number of true null hypotheses (not just for the overall null hypothesis), whereas the

Newman–Keuls allows the familywise error rate to rise as the number of true null hypotheses

increases. The Tukey test, then, provides a firm control over Type I errors, but at some loss in

power. The Newman–Keuls tries to maximize power, but with some loss in control over the

familywise error rate. A compromise, which holds the familywise error rate at abut which

also allows the critical difference between means to shrink as r(the number of means in a set)

decreases, was proposed by Ryan (1960) and subsequently modified by others.

The effect of the Newman-Keuls approach was to allow the critical values to grow as r

increases, but they actually grow too slowly to keep the familywise error rate at awhen mul-

tiple null hypotheses are true. Ryan (1960) also proposed modifying the value of afor each

step size, but in such a way that the overall familywise error rate would remain unchanged

at a. For kmeans and a step size of r, Ryan proposed using critical values of at the

ar=

a

k>r

=

ra

k

qr

s^2 i s^2 j Wr MSerror

df¿ 5

a

s^2 i

ni

1

s^2 j

nj

b

2

a

s^2 i

ni

b

2

ni 21

1

a

s^2 j

nj

b

2

nj 21

q.05(r, df¿)

12.6 Post Hoc Comparisons 393

Newman–Keuls
test