Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

324 Consistency and Limiting Distributions

Proof:Using the above results, we have

XnYn =

1

2

Xn^2 +

1

2

Yn^2 −

1

2

(Xn−Yn)^2

P

→

1

2

X^2 +

1

2

Y^2 −

1

2

(X−Y)^2 =XY.

5.1.1 SamplingandStatistics

Consider the situation where we have a random variableXwhose pdf (or pmf) is
written asf(x;θ) for an unknown parameterθ∈Ω. For example, the distribution
ofX is normal with unknown meanμand varianceσ^2 .Thenθ=(μ, σ^2 )and
Ω={θ=(μ, σ^2 ):−∞<μ<∞,σ > 0 }. As another example, the distribution
ofXis Γ(1,β), whereβ>0 is unknown. Our information consists of arandom
sampleX 1 ,X 2 ,...,XnonX; i.e.,X 1 ,X 2 ,...,Xnare independent and identically
distributed (iid) random variables with the common pdff(x;θ),θ∈Ω. We say
thatTis astatisticifTis a function of the sample; i.e.,T=T(X 1 ,X 2 ,...,Xn).
Here, we want to considerTas apoint estimatorofθ. For example, ifμis
the unknown mean ofX, then we may use as our point estimator the sample mean
X=n−^1

∑n
i=1Xi. When the sample is drawn letx^1 ,x^2 ,...,xndenote the observed
values ofX 1 ,X 2 ,...,Xn.Wecallthesevaluestherealizedvalues of the sample
and call the realized statistict=t(x 1 ,x 2 ,...,xn)apoint estimateofθ.
In Chapters 6 and 7, we discuss properties of point estimators in formal settings.
For now, we consider two properties: unbiasednessandconsistency.Wesay
that the point estimatorT forθisunbiasedifE(T)=θ. Recall in Section
2.8, we showed that the sample meanXand the sample varianceS^2 are unbiased
estimators ofμandσ^2 respectively; see equations (2.8.6) and (2.8.8). We next
consider consistency of a point estimator.

Definition 5.1.2 (Consistency). LetX be a random variable with cdfF(x, θ),
θ∈Ω.LetX 1 ,...,Xnbe a sample from the distribution ofXand letTndenote a
statistic. We sayTnis aconsistentestimator ofθif

Tn

P

→θ.

IfX 1 ,...,Xnis a random sample from a distribution with finite meanμand
varianceσ^2 , then by the Weak Law of Large Numbers, the sample mean,Xn,isa
consistent estimator ofμ.
Figure 5.1.1 displays realizations of the sample mean for samples of size 10 to
2000 in steps of 10 which are drawn from aN(0,1) distribution. The lines on the
plot encompass the intervalμ± 0 .04 forμ=0. Asnincreases, the realizations
tend to stay within this interval, verifying the consistency of the sample mean. The
R functionconsistmeanproduces this plot. Within this function, if the function
meanis changed tomediana similar plot on the estimator medXican be obtained.

Example 5.1.1(Sample Variance).LetX 1 ,...,Xndenote a random sample from
a distribution with meanμand varianceσ^2. In Example 2.8.7, we showed that the