562 Inferences About Normal Linear Models9.9 The Independence of Certain Quadratic Forms
We have previously investigated the independence of linear functions of normally
distributed variables. In this section we shall prove some theorems about the in-
dependence of quadratic forms. We shall confine our attention to normally dis-
tributed variables that constitute a random sample of sizenfrom a distribution
that isN(0,σ^2 ).
Remark 9.9.1.In the proof of the next theorem, we use the fact that ifAis an
m×nmatrix of rankn(i.e.,Ahas full column rank), then the matrixA′Ais
nonsingular. A proof of this linear algebra fact is sketched in Exercises 9.9.12 and
9.9.13.
Theorem 9.9.1(Craig).LetX′=(X 1 ,...,Xn),whereX 1 ,...,Xnare iidN(0,σ^2 )
random variables. For real symmetric matricesAandB,letQ 1 =σ−^2 X′AXand
Q 2 =σ−^2 X′BXdenote quadratic forms inX. The random variablesQ 1 andQ 2
are independent if and only ifAB= 0.
Proof:First, we obtain some preliminary results. Based on these results, the proof
follows immediately. Assume the ranks of the matricesAandBare rands,
respectively. LetΓ′ 1 Λ 1 Γ 1 denote the spectral decomposition ofA.Denotether
nonzero eigenvalues ofAbyλ 1 ,...,λr. Without loss of generality, assume that
these nonzero eigenvalues ofAare the firstrelements on the main diagonal ofΛ 1
and letΓ′ 11 be then×rmatrix whose columns are the corresponding eigenvectors.
Finally, letΛ 11 =diag{λ 1 ,...,λr}. Then we can write the spectral decomposition
ofAin either of the two ways
A=Γ′ 1 Λ 1 Γ 1 =Γ′ 11 Λ 11 Γ 11. (9.9.1)
Note that we can writeQ 1 as
Q 1 =σ−^2 X′Γ′ 11 Λ 11 Γ 11 X=σ−^2 (Γ 11 X)′Λ 11 (Γ 11 X)=W′ 1 Λ 11 W 1 , (9.9.2)
whereW 1 =σ−^1 Γ 11 X. Next, obtain a similar representation based on thesnonzero
eigenvaluesγ 1 ,...,γsofB.LetΛ 22 =diag{γ 1 ,...,γs}denote thes×sdiagonal
matrix of nonzero eigenvalues and form then×smatrixΓ′ 21 =[u 1 ···us] of corre-
sponding eigenvectors. Then we can write the spectral decomposition ofBas
B=Γ′ 21 Λ 22 Γ 21. (9.9.3)
Also, we can writeQ 2 as
Q 2 =W′ 2 Λ 22 W 2 , (9.9.4)
whereW 2 =σ−^1 Γ 21 X. LettingW′=(W′ 1 ,W′ 2 ), we have
W=σ−^1[
Γ 11
Γ 21]
X.BecauseXhas aNn( 0 ,σ^2 I) distribution, Theorem 3.5.2 shows thatWhas an
(r+s)–dimensional multivariate normal distribution with mean 0 and variance–
covariance matrix
Var (W)=[
Ir Γ 11 Γ′ 21
Γ 21 Γ′ 11 Is]. (9.9.5)