Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

392 Maximum Likelihood Methods

Next, we obtain the mles for a random sampleX 1 ,X 2 ,...,Xn. The likelihood
function is given by

L(p)=

∏n

i=1

k∏− 1

j=1

p

xji

j

⎛

⎝ 1 −

k∑− 1

j=1

pj

⎞

⎠

1 −Pkj=1−^1 xji

. (6.4.21)

Lettj=

∑n

i=1xji,forj=1,...,k−1. With simplification, the log ofLreduces to

l(p)=

k∑− 1

j=1

tjlogpj+

⎛

⎝n−

k∑− 1

j=1

tj

⎞

⎠log

⎛

⎝ 1 −

k∑− 1

j=1

pj

⎞

⎠.

The first partial ofl(p) with respect tophleads to the system of equations

∂l(p)

∂ph

=

th

ph

−

n−

∑k− 1

j=1tj

1 −

∑k− 1

j=1pj

=0,h=1,...,k− 1.

It is easily seen thatph=th/nsatisfies these equations. Hence the maximum

likelihood estimates are

̂ph=

∑n

i=1Xih

n

,h=1,...,k− 1. (6.4.22)

Each random variable

∑n

i=1Xihis binomial(n, ph) with variancenph(1−ph). There-

fore, the maximum likelihood estimates are efficient estimates.

As a final note on information, suppose the information matrix is diagonal. Then
the lower bound of the variance of thejth estimator (6.4.11) is 1/(nIjj(θ)). Because
Ijj(θ) is defined in terms of partial derivatives [see (6.4.5)] this is the information in
treating allθi, exceptθj, as known. For instance, in Example 6.4.3, for the normal
pdf the information matrix is diagonal; hence, the information forμcould have
been obtained by treatingσ^2 as known. Example 6.4.4 discusses the information
for a general location and scale family. For this general family, of which the normal
is a member, the information matrix is a diagonal matrix if the underlying pdf is
symmetric.
In the next theorem, we summarize the asymptotic behavior of the maximum
likelihood estimator of the vectorθ. It shows that the mles are asymptotically
efficient estimates.

Theorem 6.4.1.LetX 1 ,...,Xnbe iid with pdff(x;θ)forθ∈Ω. Assume the
regularity conditions hold. Then

1. The likelihood equation,
   ∂
   ∂θ
   l(θ)= 0 ,

has a solution̂θnsuch that̂θn

P

→θ.