Statistical Methods for Psychology

Exercises 241

there is a 1:1 relationship between pand retrospective power. One can be derived from

the other. Thus retrospective power offers no additional information in terms of explain-

ing nonsignificant results.

As Hoenig and Heisey (2001) argue, rather than focus our energies on calculating

retrospective power to try to learn more about what our results have to reveal, we are bet-

ter off putting that effort into calculating confidence limits on the parameter(s) or the

effect size. If, for example, we had a ttest on two independent groups with t(48) 5 1.90,

p 5 .063, we would fail to reject the null hypothesis. When we calculate retrospective

power we find it to be .46. When we calculate the 95% confidence interval on

we find 2 1.10 ## 39.1. The confidence interval tells us more about what we

are studying than does the fact that power is only .46. (Even had the difference been

slightly greater, and thus significant, the confidence interval shows that we still do not

have a very good idea of the magnitude of the difference between the population means.)

Retrospective power can be a useful tool when evaluating studies in the literature, as in

a meta-analysis, or planning future work. But retrospective power it not a useful tool for

explaining away our own non-significant results.

8.9 Writing Up the Results of a Power Analysis

We usually don’t say very much in a published study about the power of the experiment we

just ran. Perhaps that is a holdover from the fact that we didn’t even calculate power many

years ago. It is helpful, however, to add a few sentences to your Methods section that de-

scribes the power of your experiment. For example, after describing the procedures you

followed, you could say something like:

Based on the work of Jones and others (list references) we estimated that our mean dif-

ference would be approximately 8 points, with a standard deviation within each of the

groups of approximately 5. This would give us an estimated effect size of 8 11 5 .73.

We were aiming for a power estimate of .80, and to reach that level of power with our

estimated effect size, we used 30 participants in each of the two groups.

>

m 1 2m 2

m 1 2m 2

Key Terms

Power (Introduction)

Effect size (d) (8.2)

d(delta) (8.2)

Noncentrality parameter (8.3)

Harmonic mean ( ) (8.4)

Effective sample size (8.4)

A priori power (8.8)

Retrospective power (8.8)

Post hoc power (8.8)

Xh

Exercises

8.1 A large body of literature on the effect of peer pressure has shown that the mean influence

score for a scale of peer pressure is 520 with a standard deviation of 80. An investigator

would like to show that a minor change in conditions will produce scores with a mean of only

500, and he plans to run a t test to compare his sample mean with a population mean of 520.

a. What is the effect size in question?

b. What is the value of dif the size of his sample is 100?

c. What is the power of the test?

8.2 Diagram the situation described in Exercise 8.1 along the lines of Figure 8.1.

8.3 In Exercise 8.1 what sample sizes would be needed to raise power to .70, .80, and .90?