Principles of Mathematics in Operations Research

3.2 Projections and Least Squares Approximations 39

Conversely, any matrix with the above two properties represents a projection

onto the column space of A.

Proof. The projection of a projection is itself.

P^2 = A[{ATA)-^1 ATA](ATA)-^1 AT = A(ATA)~lAT = P.

We know that (S"^1 )^7 = (BT)-\ Let B = ATA.

PT = (AT)T[(ATA)-^1 }TAT = A[AT(AT)T}'^1 AT = A(ATA)~lAT = P. D

3.2.1 Orthogonal bases

Definition 3.2.4 A basis V = {VJ}"=1 is called orthonormal if

V^7 V. = (°^^J

(ortagonality)

— j (normalization)

Example 3.2.5 E — {ej}™=1 is an orthonormal basis for M", whereas X =

{xi}"=1 in Example 2.1.12 is not.

Proposition 3.2.6 If A is an m by n matrix whose columns are orthonormal

(called an orthogonal matrix), then ATA = In.

P = AAT = aiaj H h anaTn =4> x = ATb

is the least squared solution for Ax = b.

Corollary 3.2.7 An orthogonal matrix Q has the following properties:

1. QTQ = I = QQT>


2. QT = Q~\


3. QT is orthogonal.

Example 3.2.8 Suppose we project a point aT = (a,b,c) into R^2 plane.
Clearly, p — (a, b, 0) as it can be seen in Figure 3.4-

T

e\ex a =

a

0

0

i e 2 e|,Q =

P = eiej + e 2 e2 =

Pa =

"100'

010

0 0( )

a

b

c

"0"

b

0

"100"

010

0 0 0_

=

a

b

0