Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

114 Multivariate Distributions

Theorem 2.3.1.Let(X 1 ,X 2 )be a random vector such that the variance ofX 2 is

finite. Then,

(a)E[E(X 2 |X 1 )] =E(X 2 ).

(b) Var[E(X 2 |X 1 )]≤Var(X 2 ).

Proof: The proof is for the continuous case. To obtain it for the discrete case,
exchange summations for integrals. We first prove (a). Note that

E(X 2 )=

∫∞

−∞

∫∞

−∞

x 2 f(x 1 ,x 2 )dx 2 dx 1

=

∫∞

−∞

[∫∞

−∞

x 2

f(x 1 ,x 2 )

f 1 (x 1 )

dx 2

]

f 1 (x 1 )dx 1

=

∫∞

−∞

E(X 2 |x 1 )f 1 (x 1 )dx 1

= E[E(X 2 |X 1 )],

which is the first result.
Next we show (b). Consider withμ 2 =E(X 2 ),

Var(X 2 )=E[(X 2 −μ 2 )^2 ]

= E{[X 2 −E(X 2 |X 1 )+E(X 2 |X 1 )−μ 2 ]^2 }

= E{[X 2 −E(X 2 |X 1 )]^2 }+E{[E(X 2 |X 1 )−μ 2 ]^2 }

+2E{[X 2 −E(X 2 |X 1 )][E(X 2 |X 1 )−μ 2 ]}.

We show that the last term of the right-hand member of the immediately preceding

equation is zero. It is equal to

2

∫∞

−∞

∫∞

−∞

[x 2 −E(X 2 |x 1 )][E(X 2 |x 1 )−μ 2 ]f(x 1 ,x 2 )dx 2 dx 1

=2

∫∞

−∞

[E(X 2 |x 1 )−μ 2 ]

{∫∞

−∞

[x 2 −E(X 2 |x 1 )]

f(x 1 ,x 2 )

f 1 (x 1 )

dx 2

}

f 1 (x 1 )dx 1.

ButE(X 2 |x 1 ) is the conditional mean ofX 2 ,givenX 1 =x 1. Since the expression
in the inner braces is equal to

E(X 2 |x 1 )−E(X 2 |x 1 )=0,

the double integral is equal to zero. Accordingly, we have

Var(X 2 )=E{[X 2 −E(X 2 |X 1 )]^2 }+E{[E(X 2 |X 1 )−μ 2 ]^2 }.

The first term in the right-hand member of this equation is nonnegative because it

is the expected value of a nonnegative function, namely [X 2 −E(X 2 |X 1 )]^2 .Since

E[E(X 2 |X 1 )] =μ 2 , the second term is Var[E(X 2 |X 1 )]. Hence we have

Var(X 2 )≥Var[E(X 2 |X 1 )],