Fundamentals of Probability and Statistics for Engineers

To see that this procedure is plausible, we observe that the quantity

is the probability that sample X 1 ,X 2 ,...,Xn takes values in the region defined
by ( ). Given the sample values, this probability
gives a measure of likelihood that they are from the population. By choosing
a value of that maximizes L, or ln L, we in fact say that we prefer the value of
that makes as probable as possible the event that the sample values indeed
come from the population.
The extension to the case of several parameters is straightforward. In the case
of m parameters, the likelihood function becomes

and the MLEs of 1,...,m, are obtained by solving simultaneously the
system of likelihood equations

A discussion of some of the important properties associated with a maximum
likelihood estimator is now in order. Let us represent the solution of the like-
lihood equation, Equation (9.99), by

The maximum likelihood estimator for is then

The universal appeal enjoyed by maximum likelihood estimators stems from
the optimal properties they possess when the sample size becomes large. Under
mild regularity conditions imposed on the pdf or pmf of population X, two
notable properties are given below, without proof.

P ropert y 9. 1: consist ency and asy mpt ot ic efficiency.Let^ be the maximum
likelihood estimator for in pdf f(x; ) on the basis of a sample of size n. Then,
as

and

Parameter Estimation 289

L…x 1 ;x 2 ;...;xn;†dx 1 dx 2 dxn

x 1 ‡dx 1 ,x 2 ‡dx 2 ,...,xn‡dxn




L…x 1 ;...;xn; 1 ;...;m†;

j,jˆ

qlnL

q^j

ˆ 0 ; jˆ 1 ; 2 ;...;m: … 9 : 100 †

^ˆh…x 1 ;x 2 ;...;xn†: … 9 : 101 †

^ 

^ˆh…X 1 ;X 2 ;...;Xn†: … 9 : 102 †

^

 

n!1,

Ef^g!; … 9 : 103 †

varf^g! nE

qlnf…X;†

q

)) 2  1

: … 9 : 104 †