Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

4.6. Additional Comments About Statistical Tests 279

Example 4.6.4.LetX 1 ,X 2 ,...,X 10 be a random sample of sizen=10froma

Poisson distribution with meanθ. A critical region for testingH 0 :θ=0.1 against

H 1 :θ> 0 .1isgivenbyY=

∑ 10

1 Xi≥3. The statisticYhas a Poisson distribution

with mean 10θ.Thus,withθ=0.1 so that the mean ofYis 1, the significance level

of the test is

P(Y≥3) = 1−P(Y≤2) = 1−ppois(2,1) = 1− 0 .920 = 0. 080.

If, on the other hand, the critical region defined by

∑ 10

1 xi≥4 is used, the signifi-

cance level is

α=P(Y≥4) = 1−P(Y≤3) = 1−ppois(3,1) = 1− 0 .981 = 0. 019.

For instance, if a significance level of aboutα=0.05, say, is desired, most statisti-
cians would use one of these tests; that is, they would adjust the significance level
to that of one of these convenient tests. However, a significance level ofα=0. 05
can be achieved in the following way. LetW have a Bernoulli distribution with
probability of success equal to

P(W=1)=

0. 050 − 0. 019

0. 080 − 0. 019

=

31

61

Assume thatWis selected independently of the sample. Consider the rejection rule

RejectH 0 if

∑ 10

1 xi≥4orif

∑ 10

1 xi=3andW=1.

The significance level of this rule is

PH 0 (Y≥4) +PH 0 ({Y=3}∩{W=1})=PH 0 (Y≥4)

+PH 0 (Y=3)P(W=1)

=0.019 + 0. 061

31

61

=0.05;

hence, the decision rule has exactly level 0.05. The process of performing the auxil-
iary experiment to decide whether to reject or not whenY= 3 is sometimes referred
to as arandomized test.

4.6.1 Observed Significance Level,p-value

Not many statisticians like randomized tests in practice, because the use of them
means that two statisticians could make the same assumptions, observe the same
data, apply the same test, and yet make different decisions. Hence, they usually
adjust their significance level so as not to randomize. As a matter of fact, many
statisticians report what are commonly calledobserved significance levelsor
p-values(forprobability values).
A general example suffices to explain observed significance levels. LetX 1 ,...,Xn
be a random sample from aN(μ, σ^2 ) distribution, where bothμandσ^2 are unknown.