5.2 The inequalities of Markov and Chebyshev 72
Figure 5.1The figure shows a probability density, with the tails shaded indicating the
regions whereXis more thanεaway from the meanμ.
do the following quantities depend onε?
P(ˆμ≥μ+ε) and P(ˆμ≤μ−ε).
The expressions above (as a function ofε) are called thetail probabilitiesof
μˆ−μ(Fig. 5.1). Specifically, the first is called the upper tail probability and the
second the lower tail probability. Analogously,P(|μˆ−μ|≥ε)is called a two-sided
tail probability.
5.2 The inequalities of Markov and Chebyshev
The most straightforward way to bound the tails is by usingChebyshev’s
inequality, which is itself a corollary ofMarkov’s inequality. The latter is
one of the golden hammers of probability theory and so we include it for the sake
of completeness.
Lemma5.1.For any random variableXandε > 0 it holds that:
(a) (Markov):P(|X|≥ε)≤
E[|X|]
ε
.
(b) (Chebyshev):P(|X−E[X]|≥ε)≤V[X]
ε^2
.
We leave the proof of Lemma 5.1 as an exercise for the reader. By combining
(5.1)with Chebyshev’s inequality we can bound the two-sided tail directly in
terms of the variance by
P(|μˆ−μ|≥ε)≤
σ^2
nε^2
. (5.2)
This result is nice because it was so easily bought and relied on no assumptions
other than the existence of the mean and variance. The downside is that whenX
