Mathematical Methods for Physics and Engineering : A Comprehensive Guide

31.5 MAXIMUM-LIKELIHOOD METHOD

31.5.2 Transformation invariance and bias of ML estimators

An extremely useful property of ML estimators is that they areinvariantto

parameter transformations. Suppose that, instead of estimating some parameter

aof the assumed population, we wish to estimate some functionα(a)ofthe

parameter. The ML estimatorαˆ(a) is given by the value assumed by the function

α(a) at the maximum point of the likelihood, which is simply equal toα(aˆ). Thus,

we have the very convenient property

ˆα(a)=α(ˆa).

We do not have to worry about the distinction between estimatingaand estimat-

ing a function ofa.Thisisnottrue, in general, for other estimation procedures.

A company measures the duration (in minutes) of theNintervalsxi,i=1, 2 ,...,N,

between successive telephone calls received by its switchboard. Suppose that the sample

valuesxiare drawn independently from the distributionP(x|τ)=(1/τ)exp(−x/τ).Findthe

ML estimate of the parameterλ=1/τ.

This is the same problem as the first one considered in subsection 31.5.1. In terms of the
new parameterλ, the log-likelihood function is given by

lnL(x;λ)=

∑N

i=1

ln(λe−λxi)=

∑N

i=1

(lnλ−λxi).

Differentiating with respect toλand setting the result equal to zero, we have

∂lnL

∂λ

=

∑N

i=1

(

1

λ

−xi

)

=0.

Thus, the ML estimator of the parameterλis given by

λˆ=

(

1

N

∑N

i=1

xi

)− 1

= ̄x−^1. (31.75)

Referring back to (31.71), we see that, as expected, the ML estimators ofλandτare
related byλˆ=1/ˆτ.

Although this invariance property is useful it also means that, in general, ML

estimators may bebiased. In particular, one must be aware of the fact that even

ifˆais an unbiased ML estimator ofait doesnotfollow that the estimatorαˆ(a)is

also unbiased. In the limit of largeN, however, the bias of ML estimators always

tends to zero. As an illustration, it is straightforward to show (see exercise 31.8)

that the ML estimatorsτˆandˆλin the above example have expectation values

E[ˆτ]=τ and E[ˆλ]=

N

N− 1

λ. (31.76)

In fact, sinceˆτ= ̄xand the sample values are independent, the first result follows

immediately from (31.40). Thus,τˆis unbiased, butλˆ=1/τˆis biased, albeit that

the bias tends to zero for largeN.