CONFIDENCE INTERVALS VERSUS TESTS 1 13We next decide what diference between p1 and p2 we wish to detect. Note that
the actual values of p1 and p2 are not needed, only the difference. For example, if
the outcome variable was systolic blood pressure, a difference as small as 2 mmHg
would probably not be of clinical significance, so something larger would be chosen.
The size should depend on what is an important difference that we are able to detect.
A clinician might say that 5 mmHg is a sizable difference to detect, so 5 would be
used in the formula in place of p1 - p2. An estimate of cr is also needed and the
clinician may decide from past experience that cr = 15 mmHg. Then, the desired
sample size is
= 141.12
2(15)2(1.96 + .84)2 - (450)(7.84)
- (5)2 25
n=so that n = 142 are needed in each group.
If cr is unknown, so that a t test will be performed using the pooled estimate sp,
the formula above is still used. Often, we make the best estimate of cr that we can
either by looking up similar studies in the literature or from past results in the same
hospital. Alternatively, the approximate method for estimating cr from the range given
in Section 5.2.2 can be used.
Special statistical programs are available that provide sample sizes for a wide
range of statistical tests. See nQuery Advisor, PASS, Unify Pow, and Power and
Precision. Formulas for computing sample sizes for numerous tests can also be
found in Kraemer and Thiemann [1987].
8.6 CONFIDENCE INTERVALS VERSUS TESTS
In the examples discussed in this chapter and in Chapter 7, we analyzed the data using
either confidence intervals or tests of hypotheses. When are confidence intervals
generally used, and when are tests used? What are the relative advantages of the two
methods?
Most investigators run tests on the variables that are used to evaluate the success
of medical treatments and report P values. For example, if a new treatment has been
compared to a standard treatment, the readers of the results will want to know what
the chance of making a mistake is if the new treatment is adopted. Suppose that
the variable being used to evaluate two treatments is length of stay in the hospital
following treatment. Then, an investigator will probably perform a statistical test for
that variable.
For many studies, the investigators are not testing treatments, but instead, are
trying to determine if two groups are different. This was the case in the example in this
chapter when the hemoglobin levels of cyanotic and acyanotic children were analyzed.
Here, either confidence intervals or tests could be done. In general, it is sensible to
run tests when investigators have a rationale for the size of the P value that would lead
them to reject the null hypothesis and when users of the results want to know their
chance of making a type I error. For further discussion of confidence intervals and
tests of hypotheses, see van Belle et al. [2004]. Schenker and Gentleman [2001] point