Statistical Methods for Psychology

Thus we would need 8 subjects per group to have an 80% chance of rejecting if it is

false to the extent that we believe it to be.

For those readers who were disturbed by my setting dfe 5 30, it might be instructive to

calculate power for n 5 8:

From Appendix ncFfor F(4, 30; 1.71), we see that 5 .19 (by interpolation) and our

power 5 .81. This is quite close to the power 5 .80 that we sought.

In an effort to give some guidance in situations where little is known about the likely

values of parameters, Cohen (1988) has defined a small effect as 5 0.10, a medium ef-

fect as 5 0.25, and a large effect as 5 0.40. Cohen meant these to be used as a last

resort when no other estimates were possible, but it has been the general observation over

the past few years that those who calculate power most often fall back on these conven-

tions. They have tended to become starting points for a power analysis rather than a route

of last resort. I have found myself using them because I was either too lazy or too ignorant

to estimate noncentrality parameters directly, and I know of many others who fall in the

same camp. They have also become rules of thumb for deciding whether effect sizes based

on sample means should be classed as small, medium, or large. If we go back to Eysenck’s

data on recall as a function of depth of processing, we would calculate 5 .95. By Cohen’s

rule of thumb, this is a very large effect. When we looked at v^25 .393, we were saying that

depth of processing accounted for 39% of the variability in recall. Both of these statistics

are giving us useful information on the meaning of the differences.

Cohen’s 1988 book on power became the standard by which psychologists and others

calculated power, and I recommend it highly. It is still the best we have around if we want

to understand power. The terminology takes a bit of getting used to, and Cohen uses his

own tables rather than those of the noncentral F distribution, but there are many examples

and the book is well written. Bradley, Russell, and Reeve (1996) have shown that Cohen’s

power estimates tend to be conservative for more complex designs, but they are certainly

good enough for a rough estimate.

There are a number of software programs available to calculate power, and many statis-

tical analysis packages (e.g., JMP, SPSS, and DATASIM) contain the necessary routines.

I recommend G*Power, which is available free at http://www.psycho.uni-duesseldorf.de/aap/

projects/gpower/. It is easy to use, gives quick results, and lets you experiment with alterna-

tive assumptions and sample sizes. We will see an example of using G*Power shortly.

An Alternative Way to Think of Power

Imagine that we are willing to take the Conti and Musty means and variances as being a

sufficiently accurate estimate of the corresponding population means. One direct way to

estimate power (and it has become an important way in many areas), is to work with re-

sampling statistics. We start with the estimates we have and then create populations of ran-

dom numbers with those characteristics. We can then draw a large number of sets of

samples from these populations and observe what kind of sample means and variances we

actually obtain. This will give us a handle on the kinds of variability we can expect from ex-

periment to experiment, even with the kind of robust effect that Conti and Musty found. We

can go a step further and compute an Fon each set of samples and observe the variability in

f¿

f¿ f¿

f¿

b

dfe=5(7)= 35

dft= 4

f=f¿ 2 n=0.6054 28 =1.71

H 0

=7.70L8 subjects per group

=1.68^2 >.6054^2

Section 11.12 Power 351