Bandit Algorithms

2.5 Integration and expectation 30

that

∫

X 1 dPand

∫

X 2 dPare defined,

∫

(α 1 X 1 +α 2 X 2 )dPis defined and

satisfies

∫

(α 1 X 1 +α 2 X 2 ) dP=α 1

∫

X 1 dP+α 2

∫

X 2 dP. (2.5)

These two properties together tell us that wheneverX(ω) =

∑n

i=1αiI{ω∈Ai}

for somen,αi∈RandAi∈F,i= 1,...,n, then

∫

XdP=

∑

i

αiP(Ai). (2.6)

Functions of the formXare calledsimple functions.

In defining the Lebesgue integral of some random variableX, we use(2.6)as

the definition of the integral whenXis a simple function. The next step is to

extend the definition to nonnegative random variables. LetX: Ω→[0,∞) be

measurable. The idea is to approximateXfrom below using simple functions

and take the largest value that can be obtained this way.

∫

Ω

XdP= sup

{∫

Ω

hdP:his simple and 0≤h≤X

}

. (2.7)

The meaning ofU≤V for random variablesU,V is thatU(ω)≤V(ω) for all
ω∈Ω. The supremum on the right-hand side could be infinite in which case we
say the integral ofXis not defined. Whenever the integral ofXis defined we
say thatXisintegrableor, if the identity of the measurePis unclear, thatX
is integrable with respect toP.
Integrals for arbitrary random variables are defined by decomposing the
random variable into positive and negative parts. LetX : Ω→ Rbe any
measurable function. Then defineX+(ω) =X(ω)I{X(ω)> 0 }andX−(ω) =
−X(ω)I{X(ω)< 0 }so thatX(ω) =X+(ω)−X−(ω). NowX+andX−are
both nonnegative random variables called thepositiveandnegativeparts of
X. Provided that bothX+andX−are integrable we define
∫

Ω

XdP=

∫

Ω

X+dP−

∫

Ω

X−dP.

Note thatXis integrable if and only if the nonnegative-valued random variable

|X|is integrable (Exercise 2.12).

None of what we have done depends onPbeing a probability measure. The

definitions hold for any measure, though for signed measures it is necessary to

split Ω into disjoint measurable sets on which the measure is positive/negative,

an operation that is possible by theHahn decomposition theorem. We

will never need signed measures in this book, however.

A particularly interesting case is when Ω =Ris the real line,F=B(R) is

the Borelσ-algebra and the measure is theLebesgue measureλ, which is