4.1. LAWS OF LARGE NUMBERS 125
with probability densityf:
E[X] =
∫∞
0
xf(x)dx
=
∫a
0
xf(x)dx+
∫∞
a
xf(x)dx
≥
∫∞
a
xf(x)dx
≥
∫∞
a
af(x)dx
=a
∫∞
a
f(x)dx
=aP[X≥a].
(The proof for the case whereX is a purely discrete random variable is
similar with summations replacing integrals. The proof for the general case
is exactly as given withdF(x) replacingf(x)dxand interpreting the integrals
as Riemann-Stieltjes integrals.)
Lemma 4(Chebyshev’s Inequality).IfXis a random variable with finite
meanμand varianceσ^2 , then for any valuek > 0 :
P[|X−μ|≥k]≤σ^2 /k^2.
Proof. Since (X−μ)^2 is a nonnegative random variable, we can apply Markov’s
inequality (witha=k^2 ) to obtain
P
[
(X−μ)^2 ≥k^2
]
≤E
[
(X−μ)^2
]
/k^2.
But since (X−μ)^2 ≥k^2 if and only if|X−μ| ≥k, the inequality above is
equivalent to:
P[|X−μ|≥k]≤σ^2 /k^2
and the proof is complete.
Theorem 5(Weak Law of Large Numbers).LetX 1 ,X 2 ,X 3 ,...,be indepen-
dent, identically distributed random variables each with meanμand variance
σ^2. LetSn=X 1 +···+Xn. ThenSn/nconverges in probability toμ. That
is:
lim
n→∞
P[|Sn/n−μ|> ] = 0.