Mathematical Methods for Physics and Engineering : A Comprehensive Guide

STATISTICS

Suppose, we wish toestimatethe value of one of the quantitiesa 1 ,a 2 ,...,which

we will denote simply bya. Since the sample valuesxiprovide our only source of

information, any estimate ofamust be some function of thexi, i.e. some sample

statistic. Such a statistic is called anestimatorofaand is usually denoted byaˆ(x),

wherexdenotes the sample elementsx 1 ,x 2 ,...,xN.

Since an estimatorˆais a function of the sample values of the random variables

x 1 ,x 2 ,...,xN, it too must be a random variable. In other words, if a number of

random samples, each of the same sizeN, are taken from the (one-dimensional)

populationP(x|a) then the value of the estimatoraˆwill vary from one sample to

the next and in general will not be equal to the true valuea. This variation in the

estimator is described by itssampling distributionP(ˆa|a). From section 30.14, this

is given by

P(aˆ|a)daˆ=P(x|a)dNx,

wheredNxis the infinitesimal ‘volume’ inx-space lying between the ‘surfaces’

aˆ(x)=aˆandaˆ(x)=aˆ+daˆ. The form of the sampling distribution generally

depends upon the estimator under consideration and upon the form of the

population from which the sample was drawn, including, as indicated, the true

values of the quantitiesa. It is also usually dependent on the sample sizeN.

The sample valuesx 1 ,x 2 ,...,xNare drawn independently from a Gaussian distribution

with meanμand varianceσ. Suppose that we choose the sample mean ̄xas our estimator

μˆof the population mean. Find the sampling distributions of this estimator.

The sample mean ̄xis given by

̄x=

1

N

(x 1 +x 2 +···+xN),

where thexiare independent random variables distributed asxi∼N(μ, σ^2 ). From our
discussion of multiple Gaussian distributions on page 1189, we see immediately that ̄xwill
also be Gaussian distributed asN(μ, σ^2 /N). In other words, the sampling distribution of
̄xis given by

P(x ̄|μ, σ)=

1

√

2 πσ^2 /N

exp

[

−

(x ̄−μ)^2

2 σ^2 /N

]

. (31.13)

Note that the variance of this distribution isσ^2 /N.

31.3.1 Consistency, bias and efficiency of estimators

For any particular quantitya, we may in fact define any number of different

estimators, each of which will have its own sampling distribution. The quality

of a given estimatoraˆmay be assessed by investigating certain properties of its

sampling distributionP(ˆa|a). In particular, an estimatoraˆis usually judged on

the three criteria ofconsistency,biasandefficiency, each of which we now discuss.