Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

206 Some Special Distributions

3.5.3 ∗Applications...........................

In this section, we consider several applications of the multivariate normal distri-
bution. These the reader may have already encountered in an applied course in
statistics. The first isprincipal components, which results in a linear function of a
multivariate normal random vector that has independent components and preserves
the “total” variation in the problem.
Let the random vectorXhave the multivariate normal distributionNn(μ,Σ)
whereΣis positive definite. As in (3.5.8), write the spectral decomposition ofΣ
asΣ=Γ′ΛΓ. Recall that the columns,v 1 ,v 2 ,...,vn,ofΓ′are the eigenvectors
corresponding to the eigenvaluesλ 1 ,λ 2 ,...,λnthat form the main diagonal of the
matrixΛ. Assume without loss of generality that the eigenvalues are decreasing;
i.e.,λ 1 ≥λ 2 ≥ ··· ≥λn>0. Define the random vectorY=Γ(X−μ). Since
ΓΣΓ′=Λ, by Theorem 3.5.2Yhas aNn( 0 ,Λ) distribution. Hence the components
Y 1 ,Y 2 ,...,Ynare independent random variables and, fori =1, 2 ,...,n,Yihas
aN(0,λi) distribution. The random vectorYis called the vector ofprincipal
components.
We say thetotal variation, (TV), of a random vector is the sum of the variances
of its components. For the random vectorX, becauseΓis an orthogonal matrix

TV(X)=

∑n

i=1

σi^2 =trΣ=trΓ′ΛΓ=trΛΓΓ′=

∑n

i=1

λi=TV(Y).

Hence,XandYhave the same total variation.
Next, consider the first component ofY,whichisgivenbyY 1 =v 1 ′(X−μ).
This is a linear combination of the components of∑ X−μwith the property‖v 1 ‖^2 =
n
j=1v

2
1 j= 1, becauseΓ
′is orthogonal. Consider any other linear combination of
(X−μ), saya′(X−μ) such that‖a‖^2 = 1. Becausea∈Rnand{v 1 ,...,vn}forms
abasisforRn,wemusthavea=

∑n
j=1ajvj for some set of scalarsa^1 ,...,an.
Furthermore, because the basis{v 1 ,...,vn}is orthonormal

a′vi=

⎛

⎝

∑n

j=1

ajvj

⎞

⎠

′

vi=

∑n

j=1

ajv′jvi=ai.

Using (3.5.9) and the fact thatλi>0, we have the inequality

Var(a′X)=a′Σa

=

∑n

i=1

λi(a′vi)^2

=

∑n

i=1

λia^2 i≤λ 1

∑n

i=1

a^2 i=λ 1 =Var(Y 1 ). (3.5.27)

Hence,Y 1 has the maximum variance of any linear combinationa′(X−μ), such
that‖a‖= 1. For this reason,Y 1 is called thefirst principal componentofX.