Mathematical Methods for Physics and Engineering : A Comprehensive Guide

31.5 MAXIMUM-LIKELIHOOD METHOD

By substitutingσ=1/u(so thatdσ=−du/u^2 ) and integrating by parts either (N−2)/ 2
or (N−3)/2 times, we find

P(μ|x,H)∝

[

N( ̄x−μ)^2 +Ns^2

]−(N−1)/ 2

,

where we have used the fact that

∑

i(xi−μ)

(^2) =N( ̄x−μ) (^2) +Ns (^2) , ̄xbeing the sample mean
ands^2 the sample variance. We may now obtain the 95% central confidence interval by
finding the valuesμ−andμ+for which
∫μ−
−∞
P(μ|x,H)dμ=0. 025 and

∫∞

μ+

P(μ|x,H)dμ=0. 025.

The normalisation of the posterior distribution and the valuesμ−andμ+are easily
obtained by numerical integration. Substituting in the appropriate valuesN= 10, ̄x=1. 11
ands=1.01, we find the required confidence interval to be [0. 29 , 1 .97].
To obtain a confidence interval onσ, we must first obtain the corresponding marginal
posterior distribution. From (31.87), again using the fact that

∑

i(xi−μ)

(^2) =N( ̄x−μ) (^2) +Ns (^2) ,
this is given by
P(σ|x,H)∝

1

σN

exp

(

−

Ns^2

2 σ^2

)∫∞

−∞

exp

[

−

N( ̄x−μ)^2

2 σ^2

]

dμ.

Noting that the integral of a one-dimensional Gaussian is proportional toσ, we conclude
that

P(σ|x,H)∝

1

σN−^1

exp

(

−

Ns^2

2 σ^2

)

.

The 95% central confidence interval onσcan then be found in an analogous manner to
that onμ, by solving numerically the equations
∫σ−

0

P(σ|x,H)dσ=0. 025 and

∫∞

σ+

P(σ|x,H)dσ=0. 025.

We find the required interval to be [0. 76 , 2 .16].

31.5.6 Behaviour of ML estimators for largeN

As mentioned in subsection 31.3.6, in the large-sample limitN→∞, the sampling

distribution of a set of (consistent) estimatorsaˆ, whether ML or not, will tend,

in general, to a multivariate Gaussian centred on the true valuesa.Thisisa

direct consequence of the central limit theorem. Similarly, in the limitN→∞the

likelihood functionL(x;a)alsotends towards a multivariate Gaussian but one

centred on the ML estimate(s)ˆa. Thus ML estimators are alwaysasymptotically

consistent. This limiting process was illustrated for the one-dimensional case by

figure 31.5.

Thus, asNbecomes large, the likelihood function tends to the form

L(x;a)=Lmaxexp

[

−^12 Q(a,ˆa)

]

,

whereQdenotes the quadratic form

Q(a,ˆa)=(a−aˆ)TV−^1 (a−aˆ)