6.6. The EM Algorithm 409
An estimate of this expectation is the likelihood ofxibeing drawn from distribution
f 2 (x), which is given by
γi=
̂f 2 , 0 (xi)
(1−̂ )f 1 , 0 (xi)+̂f 2 , 0 (xi)
, (6.6.20)
where the subscript 0 signifies that the parameters atθ 0 are being used. Expression
(6.6.20) is intuitively evident; see McLachlan and Krishnan (1997) for more discus-
sion. Replacingwibyγiin expression (6.6.19), the M step of the algorithm is to
maximize
Q(θ|θ 0 ,x)=
∑n
i=1
[(1−γi)logf 1 (xi)+γilogf 2 (xi)]. (6.6.21)
This maximization is easy to obtain by taking partial derivatives ofQ(θ|θ 0 ,x)with
respect to the parameters. For example,
∂Q
∂μ 1
=
∑n
i=1
(1−γi)(− 1 / 2 σ^21 )(−2)(xi−μ 1 ).
Setting this to 0 and solving forμ 1 yields the estimate ofμ 1. The estimates of the
other mean and the variances can be obtained similarly. These estimates are
̂μ 1 =
∑n
∑i=1(1−γi)xi
n
i=1(1−γi)
̂σ 12 =
∑n
i=1(1−γi)(xi−μ̂^1 )
2
∑n
i=1(1−γi)
̂μ 2 =
∑n
∑i=1γixi
n
i=1γi
̂σ 22 =
∑n
i=1γi(xi−μ̂^2 )
2
∑n
i=1γi
.
Sinceγiis an estimate ofP[Wi=1|θ 0 ,x], the averagen−^1
∑n
i=1γiis an estimate
of =P[Wi= 1]. This average is our estimate of̂.
EXERCISES
6.6.1.Rao (page 368, 1973) considers a problem in the estimation of linkages in
genetics. McLachlan and Krishnan (1997) also discuss this problem and we present
their model. For our purposes, it can be described as a multinomial model with the
four categoriesC 1 ,C 2 ,C 3 ,andC 4. For a sample of sizen,letX=(X 1 ,X 2 ,X 3 ,X 4 )′
denote the observed frequencies of the four categories. Hence,n=
∑ 4
i=1Xi.The
probability model is
C 1 C 2 C 3 C 4
1
2 +
1
4 θ
1
4 −
1
4 θ
1
4 −
1
4 θ
1
4 θ