126 Chapter 4:Random Variables and Expectation
⇔
P{X=1,Y= 1 }
P{X= 1 }
>P{Y = 1 }
⇔P{Y= 1 |X= 1 }>P{Y= 1 }
That is, the covariance ofXandYis positive if the outcomeX=1 makes it more likely
thatY=1 (which, as is easily seen by symmetry, also implies the reverse).
In general, it can be shown that a positive value of Cov(X,Y) is an indication that
Ytends to increase asXdoes, whereas a negative value indicates thatYtends to decrease
asXincreases. The strength of the relationship betweenX andY is indicated by the
correlation betweenXandY, a dimensionless quantity obtained by dividing the covari-
ance by the product of the standard deviations ofXandY. That is,
Corr(X,Y)=
Cov(X,Y)
√
Var(X)Var(Y)
It can be shown (see Problem 49) that this quantity always has a value between−1 and+1.
4.8Moment Generating Functions
The moment generating functionφ(t) of the random variableXis defined for all values
tby
φ(t)=E[etX]=
∑
x
etxp(x)ifXis discrete
∫∞
−∞
etxf(x)dx ifXis continuous
We callφ(t) the moment generating function because all of the moments ofXcan be
obtained by successively differentiatingφ(t). For example,
φ′(t)=
d
dt
E[etX]
=E
[
d
dt
(etX)
]
=E[XetX]
Hence,
φ′(0)=E[X]
Similarly,
φ′′(t)=
d
dt
φ′(t)
=
d
dt
E[XetX]