Statistical Methods for Psychology

If an experimenter concerns himself with the power of a test, then he is most likely

interested in those variables governing power that are easy to manipulate. Since nis more

easily manipulated than is either or the difference ( ), and since tampering with

aproduces undesirable side effects in terms of increasing the probability of a Type I error,

discussions of power are generally concerned with the effects of varying sample size.

8.2 Effect Size

As we saw in Figures 8.1 through 8.3, power depends on the degree of overlap between

the sampling distributions under and. Furthermore, this overlap is a function of

both the distance between and and the standard error. One measure, then, of the

degree to which is false would be the distance from to expressed in terms of

the number of standard errors. The problem with this measure, however, is that it

includes the sample size (in the computation of the standard error), when in fact we will

usually wish to solve for the power associated with a given nor else for that value of n

required for a given level of power. For this reason we will take as our distance measure,

or effect size (d)

ignoring the sign of d, and incorporating nlater. Thus, dis a measure of the degree to

which and differ in terms of the standard deviation of the parent population. We see

that dis estimated independently of n, simply by estimating , , and s. In chapter 7 we

discussed effect size as the standardized difference between two means. This is the same

measure here, though one of those means is the mean under the null hypothesis. I will point

this out again when we come to comparing the means of two populations.

Estimating the Effect Size

The first task is to estimate d, since it will form the basis for future calculations. This can

be done in three ways:

1.Prior research. On the basis of past research, we can often get at least a rough approxi-

mation of d. Thus, we could look at sample means and variances from other studies and

make an informed guess at the values we might expect for and for s. In prac-

tice, this task is not as difficult as it might seem, especially when you realize that a

rough approximation is far better than no approximation at all.

2.Personal assessment of how large a difference is important. In many cases, an investi-

gator is able to say, I am interested in detecting a difference of at least 10 points between

and. The investigator is essentially saying that differences less than this have no

important or useful meaning, whereas greater differences do. (This is particularly com-

mon in biomedical research, where we are interesting in decreasing cholesterol, for ex-

ample, by a certain amount, and have no interest in smaller changes.) Here we are given

the value of directly, without needing to know the particular values of and

. All that remains is to estimate sfrom other data. As an example, the investigator
might say that she is interested in finding a procedure that will raise scores on the Grad-
uate Record Exam by 40 points above normal. We already know that the standard devi-
ation for this test is 100. Thus d 5 40/100 5 .40. If our hypothetical experimenter says
instead that she wants to raise scores by four-tenths of a standard deviation, she would
be giving us ddirectly.

m 0

m 1 2m 0 m 1

m 1 m 0

m 1 2m 0

m 1 m 0

m 1 m 0

d=

m 1 2m 0

s

H 0 m 1 m 0

m 0 m 1

H 0 H 1

s^2 m 0 2m 1

Section 8.2 Effect Size 229

effect size (d)