Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

372 Maximum Likelihood Methods

Corollary 6.2.3.Under the assumptions of Theorem 6.2.2,

√

n(θ̂n−θ 0 )=

1

I(θ 0 )

1

√

n

∑n

i=1

∂logf(Xi;θ 0 )

∂θ

+Rn, (6.2.31)

whereRn→P 0.
The proof is just a rearrangement of equation (6.2.20) and the ensuing results in
the proof of Theorem 6.2.2.
Example 6.2.6(Example 6.2.4, Continued).LetX 1 ,...,Xnbe a random sample
having the common pdf (6.2.14). Recall that I(θ)=θ−^2 and that the mle is
̂θ=−n/∑ni=1logXi. Hence,θ̂is approximately normally distributed with meanθ

and varianceθ^2 /n. Based on this, an approximate (1−α)100% confidence interval
forθis
θ̂±zα/ 2

θ̂

√

n

.

Recall that we were able to obtain the exact distribution ofθ̂in this case. As
Exercise 6.2.12 shows, based on this distribution of̂θ, an exact confidence interval
forθcan be constructed.
In obtaining the mle ofθ, we are often in the situation of Example 6.1.2; that
is, we can verify the existence of the mle, but the solution of the equationl′(̂θ)=
0 cannot be obtained in closed form. In such situations, numerical methods are
used. One iterative method that exhibits rapid (quadratic) convergence is Newton’s
method. The sketch in Figure 6.2.1 helps recall this method. Supposêθ(0)is an
initial guess at the solution. The next guess (one-step estimate) is the pointθ̂(1),
which is the horizontal intercept of the tangent line to the curvel′(θ)atthepoint

(θ̂(0),l′(θ̂(0))). A little algebra finds

̂θ(1)=̂θ(0)−l

′(̂θ(0))

l′′(̂θ(0))

. (6.2.32)

We then substituteθ̂(1)for̂θ(0)and repeat the process. On the figure, trace the
second step estimatêθ(2); the process is continued until convergence.

Example 6.2.7(Example 6.1.2, continued).Recall Example 6.1.2, where the ran-
dom sampleX 1 ,...,Xnhas the common logistic density

f(x;θ)=

exp{−(x−θ)}

(1 + exp{−(x−θ)})^2

, −∞<x<∞,−∞<θ<∞. (6.2.33)

We showed that the likelihood equation has a unique solution, though it cannot be
be obtained in closed form. To use formula (6.2.32), we need the first and second
partial derivatives ofl(θ) and an initial guess. Expression (6.1.9) of Example 6.1.2
gives the first partial derivative, from which the second partial is

l′′(θ)=− 2

∑n

i=1

exp{−(xi−θ)}

(1 + exp{−(xi−θ)})^2

.