1.10. Important Inequalities 81Definition 1.10.1.Afunctionφdefined on an interval(a, b),−∞ ≤a<b≤∞,
is said to be aconvexfunction if for allx, yin(a, b)and for all 0 <γ< 1 ,
φ[γx+(1−γ)y]≤γφ(x)+(1−γ)φ(y). (1.10.4)We sayφisstrictly convexif the above inequality is strict.
Depending on the existence of first or second derivatives of φ, the following
theorem can be proved.
Theorem 1.10.4.Ifφis differentiable on(a, b),then
(a)φis convex if and only ifφ′(x)≤φ′(y),for alla<x<y<b,(b)φis strictly convex if and only ifφ′(x)<φ′(y),for alla<x<y<b.Ifφis twice differentiable on(a, b),then
(a)φis convex if and only ifφ′′(x)≥ 0 ,for alla<x<b,(b)φis strictly convex ifφ′′(x)> 0 ,for alla<x<b.Of course, the second part of this theorem follows immediately from the first
part. While the first part appeals to one’s intuition, the proof of it can be found in
most analysis books; see, for instance, Hewitt and Stromberg (1965). A very useful
probability inequality follows from convexity.
Theorem 1.10.5(Jensen’s Inequality).Ifφis convex on an open intervalIand
Xis a random variable whose support is contained inIand has finite expectation,
then
φ[E(X)]≤E[φ(X)]. (1.10.5)
Ifφis strictly convex, then the inequality is strict unlessXis a constant random
variable.
Proof:For our proof we assume thatφhas a second derivative, but in general only
convexity is required. Expandφ(x) into a Taylor series aboutμ=E[X]oforder2:
φ(x)=φ(μ)+φ′(μ)(x−μ)+φ′′(ζ)(x−μ)^2
2,whereζis betweenxandμ.^9 Because the last term on the right side of the above
equation is nonnegative, we have
φ(x)≥φ(μ)+φ′(μ)(x−μ).Taking expectations of both sides leads to the result. The inequality is strict if
φ′′(x)>0, for allx∈(a, b), providedXis not a constant.
(^9) See, for example, the discussion on Taylor series inMathematical Commentsreferenced in the
Preface.