Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

410 Maximum Likelihood Methods

where the parameterθsatisfies 0≤θ≤1. In this exercise, we obtain the mle ofθ.

(a)Show that likelihood function is given by

L(θ|x)=

n!

x 1 !x 2 !x 3 !x 4!

[

1

2

+

1

4

θ

]x 1 [

1

4

−

1

4

θ

]x 2 +x 3 [

1

4

θ

]x 4

. (6.6.22)

(b)Show that the log of the likelihood function can be expressed as a constant

(not involving parameters) plus the term

x 1 log[2 +θ]+[x 2 +x 3 ] log[1−θ]+x 4 logθ.

(c)Obtain the partial derivative with respect toθof the last expression, set the

result to 0, and solve for the mle. (This will result in a quadratic equation

that has one positive and one negative root.)

6.6.2. In this exercise, we set up an EM algorithm to determine the mle for the
situation described in Exercise 6.6.1. Split categoryC 1 into the two subcategories
C 11 andC 12 with probabilities 1/2andθ/4, respectively. LetZ 11 andZ 12 denote
the respective “frequencies.” ThenX 1 =Z 11 +Z 12. Of course, we cannot observe
Z 11 andZ 12 .LetZ=(Z 11 ,Z 12 )′.

(a)Obtain the complete likelihoodLc(θ|x,z).

(b)Using the last result and (6.6.22), show that the conditional pmfk(z|θ,x)is

binomial with parametersx 1 and probability of successθ/(2 +θ).

(c)Obtain the E step of the EM algorithm given an initial estimatêθ(0)ofθ.

That is, obtain

Q(θ|̂θ(0),x)=Eθb(0)[logLc(θ|x,Z)|̂θ(0),x].

Recall that this expectation is taken using the conditional pmfk(z|̂θ(0),x).

Keep in mind the next step; i.e., we need only terms that involveθ.

(d)For the M step of the EM algorithm, solve the equation∂Q(θ|̂θ(0),x)/∂θ=0.

Show that the solution is

θ̂(1)= x^1

̂θ(0)+2x 4 +x 4 θ̂(0)

nθ̂(0)+2(x 2 +x 3 +x 4 )

. (6.6.23)

6.6.3. For the setup of Exercise 6.6.2, show that the following estimator ofθis
unbiased:
θ ̃=n−^1 (X 1 −X 2 −X 3 +X 4 ). (6.6.24)

6.6.4.Rao (page 368, 1973) presents data for the situation described in Exercise
6.6.1. The observed frequencies arex= (125, 18 , 20 ,34)′.

(a)Using computational packages (for example, R), with (6.6.24) as the initial

estimate, write a program that obtains the stepwise EM estimatesθ̂(k).