Introduction to Probability and Statistics for Engineers and Scientists

7.2Maximum Likelihood Estimators 231

In determining the maximizing value ofθ, it is often useful to use the fact that
f(x 1 ,...,xn|θ) and log[f(x 1 ,...,xn|θ)]have their maximum at the same value ofθ.
Hence, we may also obtainθˆby maximizing log[f(x 1 ,...,xn|θ)].

EXAMPLE 7.2a Maximum Likelihood Estimator of a Bernoulli ParameterSupposethatninde-
pendent trials, each of which is a success with probabilityp, are performed. What is the
maximum likelihood estimator ofp?

SOLUTION The data consist of the values ofX 1 ,...,Xnwhere

Xi=

{

1 if trialiis a success

0 otherwise

Now

P{Xi= 1 }=p= 1 −P{Xi= 0 }

which can be succinctly expressed as

P{Xi=x}=px(1−p)^1 −x, x=0, 1

Hence, by the assumed independence of the trials, the likelihood (that is, the joint
probability mass function) of the data is given by

f(x 1 ,...,xn|p)=P{X 1 =x 1 ,...,Xn=xn|p}

=px^1 (1−p)^1 −x^1 ···pxn(1−p)^1 −xn

=p

n 1 xi

(1−p)n−

1 nxi

, xi=0, 1, i=1,...,n

To determine the value ofpthat maximizes the likelihood, first take logs to obtain

logf(x 1 ,...,xn|p)=

∑n

1

xilogp+

(

n−

∑n

1

xi

)

log(1−p)

Differentiation yields

d

dp

logf(x 1 ,...,xn|p)=

∑n

1

xi

p

−

(

n−

∑n

1

xi

)

1 −p