Mathematical Methods for Physics and Engineering : A Comprehensive Guide

31.5 MAXIMUM-LIKELIHOOD METHOD

a a

a a

L(x;a) L(x;a)

L(x;a) L(x;a)

aˆ aˆ

ˆa aˆ

(a) (b)

(c) (d)

Figure 31.6 Typical shapes of one-dimensional likelihood functionsL(x;a)

encountered in practice, when, for illustration purposes, it is assumed that the

parameterais restricted to the range zero to infinity. The ML estimator in

the various cases occurs at: (a) the only stationary point; (b) one of several

stationary points; (c) an end-point of the allowed parameter range that is not

a stationary point (although stationary points do exist); (d) an end-point of

the allowed parameter range in which no stationary point exists.

31.5.1 The maximum-likelihood estimator

Since the likelihood functionL(x;a) gives the probability density associated with

any particular set of values of the parametersa, our best estimateaˆof these

parameters is given by the values ofafor whichL(x;a) is a maximum. This is

called themaximum-likelihood estimator(or ML estimator).

In general, the likelihood function can have a complicated shape when con-

sidered as a function ofa, particularly when the dimensionality of the space of

parametersa 1 ,a 2 ,...,aMis large. It may be that the values of some parameters

are either known or assumed in advance, in which case the effective dimension-

ality of the likelihood function is reduced accordingly. However, even when the

likelihood depends on just a single parametera(either intrinsically or as the

result of assuming particular values for the remaining parameters), its form may

be complicated when the sample sizeNis small. Frequently occurring shapes of

one-dimensional likelihood functions are illustrated in figure 31.6, where we have

assumed, for definiteness, that the allowed range of the parameterais zero to

infinity. In each case, the ML estimateˆais also indicated. Of course, the ‘shape’ of

higher-dimensional likelihood functions may be considerably more complicated.

In many simple cases, however, the likelihood functionL(x;a) has a single