6.2. Rao–Cram ́er Lower Bound and Efficiency 369
Theorem 6.2.2.AssumeX 1 ,...,Xnare iid with pdff(x;θ 0 )forθ 0 ∈Ωsuch that
the regularity conditions (R0)–(R5) are satisfied. Suppose further that the Fisher
information satisfies 0 <I(θ 0 )<∞. Then any consistent sequence of solutions of
the mle equations satisfies
√
n(̂θ−θ 0 )
D
→N
(
0 ,
1
I(θ 0 )
)
. (6.2.18)
Proof: Expanding the functionl′(θ)intoaTaylorseriesoforder2aboutθ 0 and
evaluating it atθ̂n,weget
l′(θ̂n)=l′(θ 0 )+(θ̂n−θ 0 )l′′(θ 0 )+
1
2
(θ̂n−θ 0 )^2 l′′′(θn∗), (6.2.19)
whereθn∗is betweenθ 0 andθ̂n.Butl′(θ̂n) = 0. Hence, rearranging terms, we obtain
√
n(̂θn−θ 0 )=
n−^1 /^2 l′(θ 0 )
−n−^1 l′′(θ 0 )−(2n)−^1 (̂θn−θ 0 )l′′′(θ∗n)
. (6.2.20)
By the Central Limit Theorem,
1
√
n
l′(θ 0 )=
1
√
n
∑n
i=1
∂logf(Xi;θ 0 )
∂θ
D
→N(0,I(θ 0 )), (6.2.21)
because the summands are iid with Var(∂logf(Xi;θ 0 )/∂θ)=I(θ 0 )<∞.Also,by
the Law of Large Numbers,
−
1
n
l′′(θ 0 )=−
1
n
∑n
i=1
∂^2 logf(Xi;θ 0 )
∂θ^2
→P I(θ
0 ). (6.2.22)
To complete the proof then, we need only show that the second term in the
denominator of expression (6.2.20) goes to zero in probability. Becausêθn−θ 0 →P 0
by Theorem 5.2.7, this follows provided thatn−^1 l′′′(θ∗n) is bounded in probability.
Letc 0 be the constant defined in condition (R5). Note that|θ̂n−θ 0 |<c 0 implies
that|θ∗n−θ 0 |<c 0 , which in turn by condition (R5) implies the following string of
inequalities:
∣
∣
∣
∣−
1
n
l′′′(θ∗n)
∣
∣
∣
∣≤
1
n
∑n
i=1
∣
∣
∣
∣
∂^3 logf(Xi;θ)
∂θ^3
∣
∣
∣
∣≤
1
n
∑n
i=1
M(Xi). (6.2.23)
By condition (R5),Eθ 0 [M(X)]<∞; hence,^1 n
∑n
i=1M(Xi)
P
→Eθ 0 [M(X)], by the
Law of Large Numbers. For the bound, we select 1 +Eθ 0 [M(X)]. Let>0be
given. ChooseN 1 andN 2 so that
n≥N 1 ⇒ P[|̂θn−θ 0 |<c 0 ]≥ 1 −
2
(6.2.24)
n≥N 2 ⇒ P
[∣
∣
∣
∣
∣
1
n
∑n
i=1
M(Xi)−Eθ 0 [M(X)]
∣
∣
∣
∣
∣
< 1
]
≥ 1 −
2
. (6.2.25)