5.6 Exercises 80(a)MXis convex and in particular dom(MX) is an interval containing zero.
(b)MX(λ)≥eλE[X]for allλ∈dom(MX).
(c)For anyλ in the interior of dom(MX),MX is infinitely many times
differentiable.
(d)Let MX(k)(λ) = d
k
dλkMX(λ). Then, for λin the interior of dom(MX),
M(k)(λ) =E[
Xkexp(λX)]
.
(e)Assuming 0 is in the interior ofdom(MX),MX(k)(0) =E[
Xk]
(hence the
name ofMX).
(f)ψXis convex (that is,MXis log-convex).Hint For part (a) use the convexity ofx7→ex.
5.10(Large deviation theory) LetX,X 1 ,X 2 ,...,Xn be a sequence of
independent and identically distributed random variables with zero mean and
moment generating functionMXwith dom(MX) =R. Let ˆμn=n^1∑n
t=1Xt.(a) Show that for anyε >0,
1
nlogP(ˆμn≥ε)≤−ψ∗
X(ε) =−supλ (λε−logMX(λ)). (5.9)(b) Letσ^2 =V[X]. The central limit theorem says that for anyx∈R,
nlim→∞P(
μˆn√
n
σ^2≥x)
= Φ(x),where Φ(x) =√^12 π∫x
−∞exp(−y(^2) /2)dyis the cumulative distribution of the
standard Gaussian. LetZbe a random variable distributed like a standard
Gaussian. A careless application of this result might suggest that
nlim→∞
1
nlogP(ˆμn≥ε)
= lim?
n→∞1
nlogP(
Z≥ε√
n
σ^2)
.
Evaluate the right-hand side and explain why the question-marked equality
doesnothold.As it happens, the inequality in(5.9) may be replaced by an equality
asn→ ∞. The assumption that the moment-generating function exists
everywhere may be relaxed significantly. We refer the interested reader to
the classic text by Dembo and Zeitouni [2009]. The functionψ∗Xis called the
Legendre transform,convex conjugateorFenchel dualof the convex
functionψX. Convexity and the Fenchel dual will play a role in some of the
later chapters and will be discussed in more detail in Chapter 26 and later.5.11(Hoeffding’s lemma) Suppose thatXis zero-mean andX∈[a,b] almost
surely for constantsa < b.(a) Show thatXis (b−a)/2-subgaussian.