STATISTICS
31.5.3 Efficiency of ML estimatorsWe showed in subsection 31.3.2 that Fisher’s inequality puts a lower limit on the
varianceV[aˆ] of any estimator of the parametera. Under our hypothesisHon
p. 1255, the functional form of the population is given by the likelihood function,
i.e.P(x|a,H)=L(x;a). Thus, if this hypothesis is correct, we may replacePby
Lin Fisher’s inequality (31.18), which then reads
V[aˆ]≥(
1+∂b
∂a) 2 /
E[
−∂^2 lnL
∂a^2]
,wherebis the bias in the estimatoraˆ. We usually denote the RHS byVmin.
An important property of ML estimators is thatifthere exists an efficientestimatorˆaeff,i.e.oneforwhichV[aˆeff]=Vmin,thenitmustbe the ML estimator
or some function thereof. This is easily shown by replacingPbyLin the proof
of Fisher’s inequality given in subsection 31.3.2. In particular, we note that the
equality in (31.22) holds only ifh(x)=cg(x), wherecis a constant. Thus, if an
efficient estimatoraˆeffexists, this is equivalent to demanding that
∂lnL
∂a=c[aˆeff−α(a)].Now, the ML estimatorˆaMLis given by
∂lnL
∂a∣
∣
∣
∣
a=aˆML=0 ⇒ c[aˆeff−α(ˆaML)] = 0,which, in turn, implies thataˆeffmust be some function ofˆaML.
Show that the ML estimatorτˆgiven in (31.71) is an efficient estimator of the parameterτ.As shown in (31.70), the log-likelihood function in this case is
lnL(x;τ)=−∑N
i=1(
lnτ+xi
τ)
.
Differentiating twice with respect toτ, we find
∂^2 lnL
∂τ^2=
∑N
i=1(
1
τ^2−
2 xi
τ^3)
=
N
τ^2(
1 −
2
τN∑N
i=1xi)
, (31.77)
and so the expectation value of this expression is
E[
∂^2 lnL
∂τ^2]
=
N
τ^2(
1 −
2
τE[xi])
=−
N
τ^2,
where we have used the fact thatE[x]=τ. Settingb= 0 in (31.18), we thus find that for
anyunbiased estimator ofτ,
V[τˆ]≥τ^2
N.
From (31.76), we see that the ML estimatorˆτ=
∑
ixi/Nis unbiased. Moreover, using
the fact thatV[x]=τ^2 , it follows immediately from (31.40) thatV[τˆ]=τ^2 /N. Thusτˆis a
minimum-variance estimator ofτ.