Statistical Methods for Psychology

Applying this to our data from Table 11.2, we have

The estimate of in this case (.393) is noticeably less than the estimate of 5 .447,

reflecting the fact that the latter is more biased.

We have discussed two measures of the degree of association between the dependent

and independent variables. These are only two of the many approaches that have been sug-

gested. In general, is probably the best. Fowler (1985) presents evidence on the bias of

six different estimates and shows that performs well.

Aside from their concern about whether one statistic is more or less biased than an-

other, researchers have raised questions regarding the interpretation of magnitude of ef-

fect measures in general, especially those classed as r-family measures. Rosenthal and

Rubin (1982) present an interesting argument that quite small values of r^2 (the squared

correlation coefficient) can represent important and dramatic effects. O’Grady (1982)

presents several arguments why magnitude-of-effect measures may not be good meas-

ures of whatever it is we mean by “importance.” Even an important variable may, for sev-

eral reasons, account for small percentages of variation, and, more commonly, a large

value of may simply mean that we have studied a trivial variable (such as the difference

in height between elementary-school children and college students). (Even if not for what

O’Grady says about the magnitude of effect, his excellent paper is worth reading for what

it has to say about the psychometric and methodological considerations behind all the stud-

ies psychologists run.) Lane and Dunlap (1978) raise some important reservations about

the routine reporting of magnitude measures and their interpretation in light of the fact that

journals mainly publish studies with significant results. Finally, Cohen (1973) outlines

some important considerations in the calculation and interpretation of magnitude measures.

Although Cohen is primarily concerned with factorial designs (to be discussed in Chapter 13),

the philosophy behind his comments is relevant even here. All the papers cited are clear

and readable, and I recommend them.

d-Family Measures of Effect Size

I will have more to say about d-family measures of effect size in the next chapter, but here

I want to briefly discuss a measure favored by Steiger (2004) called the root-mean-square

standardized effect (RMSSE).It is based on a logical measure of group differences and

applies to the case of multiple groups. Moreover it is nearly equivalent to the effect size

that we will use in calculating power.

One measure of how much a particular group mean deviated from the overall grand

mean would be

Notice that this is simply a standardized difference between a specific mean and

the grand mean, and is similar to, though not quite the same as, the dthat we saw in

Chapter 7. It is logical to average these measures over all groups, but we will need to

square them first or the average would come out to be zero. This gives us a measure that

can be written as

d=

B

a

1

k 21

baa

mj2m

s b

2

dj=

(mj2m)

s

h^2

v^2

v^2

v^2 h^2

v^2 =

SStreat 2 (k 2 1) MSerror

SStotal 1 MSerror

=

351.52 2 4(9.67)

786.82 1 9.67

=

312.84

796.49

=.393

Section 11.11 The Size of an Experimental Effect 347

root-mean-
square
standardized
effect (RMSSE)