Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

6.2. Rao–Cram ́er Lower Bound and Efficiency 369

Theorem 6.2.2.AssumeX 1 ,...,Xnare iid with pdff(x;θ 0 )forθ 0 ∈Ωsuch that
the regularity conditions (R0)–(R5) are satisfied. Suppose further that the Fisher
information satisfies 0 <I(θ 0 )<∞. Then any consistent sequence of solutions of
the mle equations satisfies
√
n(̂θ−θ 0 )
D
→N

(

0 ,

1

I(θ 0 )

)

. (6.2.18)

Proof: Expanding the functionl′(θ)intoaTaylorseriesoforder2aboutθ 0 and
evaluating it atθ̂n,weget

l′(θ̂n)=l′(θ 0 )+(θ̂n−θ 0 )l′′(θ 0 )+

1

2

(θ̂n−θ 0 )^2 l′′′(θn∗), (6.2.19)

whereθn∗is betweenθ 0 andθ̂n.Butl′(θ̂n) = 0. Hence, rearranging terms, we obtain

√

n(̂θn−θ 0 )=

n−^1 /^2 l′(θ 0 )

−n−^1 l′′(θ 0 )−(2n)−^1 (̂θn−θ 0 )l′′′(θ∗n)

. (6.2.20)

By the Central Limit Theorem,

1

√

n

l′(θ 0 )=

1

√

n

∑n

i=1

∂logf(Xi;θ 0 )

∂θ

D

→N(0,I(θ 0 )), (6.2.21)

because the summands are iid with Var(∂logf(Xi;θ 0 )/∂θ)=I(θ 0 )<∞.Also,by
the Law of Large Numbers,

−

1

n

l′′(θ 0 )=−

1

n

∑n

i=1

∂^2 logf(Xi;θ 0 )

∂θ^2

→P I(θ

0 ). (6.2.22)

To complete the proof then, we need only show that the second term in the

denominator of expression (6.2.20) goes to zero in probability. Becausêθn−θ 0 →P 0
by Theorem 5.2.7, this follows provided thatn−^1 l′′′(θ∗n) is bounded in probability.

Letc 0 be the constant defined in condition (R5). Note that|θ̂n−θ 0 |<c 0 implies
that|θ∗n−θ 0 |<c 0 , which in turn by condition (R5) implies the following string of
inequalities:
∣
∣
∣
∣−

1

n

l′′′(θ∗n)

∣

∣

∣

∣≤

1

n

∑n

i=1

∣

∣

∣

∣

∂^3 logf(Xi;θ)

∂θ^3

∣

∣

∣

∣≤

1

n

∑n

i=1

M(Xi). (6.2.23)

By condition (R5),Eθ 0 [M(X)]<∞; hence,^1 n

∑n

i=1M(Xi)

P
→Eθ 0 [M(X)], by the
Law of Large Numbers. For the bound, we select 1 +Eθ 0 [M(X)]. Let>0be
given. ChooseN 1 andN 2 so that

n≥N 1 ⇒ P[|̂θn−θ 0 |<c 0 ]≥ 1 −

2

(6.2.24)

n≥N 2 ⇒ P

[∣

∣

∣

∣

∣

1

n

∑n

i=1

M(Xi)−Eθ 0 [M(X)]

∣

∣

∣

∣

∣

< 1

]

≥ 1 −

2

. (6.2.25)