Statistical Methods for Psychology

This example illustrates the use of repeated sampling to directly investigate the power of a

test, the variability of sample means over replication, and the meaning of the noncentrality

parameter. It also shows you graphically the idea of power looked at from the point of view of

the sampling distribution of a statistic (in this case, F). You even have the opportunity to see a

Type II error in action, because the eighth case in Table 11.7, whose probability was greater

than .05 under H 0 , is a Type II error. I took this digression to viewing power in terms of resam-

pling because more of the statistical software that is written today makes calculations of power,

confidence limits, and other statistics using the same kind of resampling statistics that we have

used here, especially when direct calculation would be difficult or impossible.

G*Power

Having recommended G*Power as an excellent program for calculating power, I have used it

to produce the following printout. Because the software makes it easy for me to deal with un-

equal sample sizes, I have used the actual sample sizes from the Conti and Musty experiment.

(However G*Power bases its calculations on the average sample size.) The screen on the right

in Exhibit 11.2 shows the results of calculating the effect size. I have specified that I want

power for an analysis of variance, and have entered the means and sample sizes for the five

groups. The program automatically computes the effect size when I click on the “Calculate

and transfer” button. In this case it is 0.6092, which is close to the same answer that we calcu-

lated earlier with sample sizes of 10. I then clicked on the “Calculate” button and the program

moved to the left window to calculate power. I requested that it calculate post-hoc power

because I am using the actual sample means and error term from the Conti and Musty data.^11

You will notice that the calculated power is .9036, which is higher than our calculation.

But remember that we had to use 30 dffor the error term in our calculation because the

tables of the noncentral Fdistribution did not allow us to use the true value of 45 df.

Koele (1982) presents methods for calculating the power of random models. Random

models, while not particularly common in a one-way layout, are more common in higher

order designs and present particular problems because they generally have a low level of power.

For these models, two random processes are involved—random sampling of participants

Section 11.12 Power 353

0 20

Value of F

Proportion

0.09

0.08

0.07

0.06

0.05

0.04

0.03

0.02

0.01

0

4 8 12 16

Figure 11.6 Empirical sampling distribution of Fwhen noncentrality parameter

equals 5.838.

(^11) If I had been writing this software I would not have used the phrase “post-hoc power” here because it conveys
different meanings to different people. What I am really doing is making parameter estimates from a previous
study and using those estimates to calculate power. That is a very valid approach even among those who decry
what is often meant as “post-hoc power.”