Mathematical Methods for Physics and Engineering : A Comprehensive Guide

31.3 ESTIMATORS AND SAMPLING DISTRIBUTIONS

Consistency

An estimatoraˆisconsistentif its value tends to the true valueain the large-sample

limit, i.e.

lim

N→∞

ˆa=a.

Consistency is usually a minimum requirement for a useful estimator. An equiv-

alent statement of consistency is that in the limit of largeNthe sampling

distributionP(aˆ|a) of the estimator must satisfy

lim

N→∞

P(ˆa|a)→δ(ˆa−a).

Bias

The expectation value of an estimatoraˆis given by

E[aˆ]=

∫

aPˆ (ˆa|a)daˆ=

∫

aˆ(x)P(x|a)dNx, (31.14)

where the second integral extends over all possible values that can be taken by

the sample elementsx 1 ,x 2 ,...,xN. This expression gives the expected mean value

ofaˆfrom an infinite number of samples, each of sizeN.Thebiasof an estimator

ˆais then defined as

b(a)=E[aˆ]−a. (31.15)

We note that the biasbdoes not depend on the measured sample values

x 1 ,x 2 ,...,xN. In general, though, it will depend on the sample sizeN, the func-

tional form of the estimatoraˆand, as indicated, on the true propertiesaof

the population, including the true value ofaitself. Ifb=0thenˆais called an

unbiasedestimator ofa.

An estimatoraˆis biased in such a way thatE[aˆ]=a+b(a),wherethebiasb(a)is given

by(b 1 −1)a+b 2 andb 1 andb 2 are known constants. Construct an unbiased estimator ofa.

Let us first writeE[ˆa] is the clearer form

E[aˆ]=a+(b 1 −1)a+b 2 =b 1 a+b 2.

The task of constructing an unbiased estimator is now trivial, and an appropriate choice
isaˆ′=(aˆ−b 2 )/b 1 , which (as required) has the expectation value

E[aˆ′]=

E[ˆa]−b 2

b 1

=a.

Efficiency

The variance of an estimator is given by

V[aˆ]=

∫

(ˆa−E[aˆ])^2 P(ˆa|a)daˆ=

∫

(aˆ(x)−E[ˆa])^2 P(x|a)dNx

(31.16)