committing a Type 2 error is not immediately specifiable except that we can say
that it is related inversely to the significance level for rejecting the null hypothesis.
The two kinds of error cannot be minimized simultaneously, except by increasing
the sample size. Hence, we need a compromise level of significance that will pro-
vide an acceptably small chance of rejecting a factual null hypothesis, but which is
not so small as to generate too large a chance of committing a Type 2 error.
Experience has indicated that a 5% chance of rejecting the null hypothesis when
it is true provides reasonable insurance against both kinds of error. We therefore
conventionally specify the 5% probability as our significance level, although that
level is essentially arbitrary and little more than a gentlemen’s agreement. What
is not arbitrary is that the hypothesis to be tested and the level of significance at
which the hypothesis is rejected must be decided upon beforethe data are examined
and preferably before they are collected. Otherwise the whole logic of testing is
violated.
Our standard statistical tests concentrate on minimizing Type 1 errors. The extent
to which they minimize Type 2 errors is called power. Depending on context, avoid-
ance of Type 2 errors may be more important than ensuring the warranted rejection
of the null hypothesis.Converting a statistical result back into a biological conclusion is not at all straight-
forward. The classical null hypothesis method is at its best when testing whether a
treatment has an effect, the treatment representing a cost and the response a benefit.
An example might be supplementary feeding to increase the clutch size of a game
bird. Here the feeding costs money and time, and we will use it operationally only
if an adequate response is clearly demonstrated. First, the null hypothesis must be
rejected, and then the difference in response between experimental control and treat-
ment must be evaluated to determine whether the cost of the treatment is justified
by the size of the response in fecundity. If the null hypothesis (no effect of treat-
ment) is not rejected we are simply back where we started and no harm is done.
Type 1 and Type 2 errors are both possible, both are inconvenient, but neither is
catastrophic. A Type 1 error leads to unnecessary expense until the mistake is
identified; a Type 2 error results in a small sacrifice in the potential fecundity of the
game bird population.
Null hypothesis testing is less effective and efficient when the treatment itself is a
benefit and the lack of treatment is itself a cost. Suppose a marine fish stock appears
to be declining although there is considerable year-to-year variation in the index of
abundance used: catch per unit effort. Further, there are good reasons to suspect that
the fishing itself is heavy enough to precipitate a decline. The null hypothesis is that
fishing has no effect on population size. In this case the failure to reject the “no-
effect” null hypothesis is not sufficient reason to operate on the assumption that the
fishing is having no effect. At the very least one would first want to know something
about the power of the test. In this case the cost of making a Type 2 mistake greatly
outweighs the benefit of getting it right. The effect of continuing to fish when one
should have stopped could be disastrous and irreversible, whereas unnecessary ces-
sation of fishing results only in a temporary cost until fishing resumes. This is an
asymmetry of risk. It is particularly prevalent in work on endangered species where
an error can result in extinction. Asymmetry of risk demands conservative interpre-
tation of statistical results.EXPERIMENTAL MANAGEMENT 271
16.3.2Asymmetry of
riskWECC16 18/08/2005 14:47 Page 271