3.2 Projections and Least Squares Approximations 39
Conversely, any matrix with the above two properties represents a projection
onto the column space of A.
Proof. The projection of a projection is itself.
P^2 = A[{ATA)-^1 ATA](ATA)-^1 AT = A(ATA)~lAT = P.
We know that (S"^1 )^7 = (BT)-\ Let B = ATA.
PT = (AT)T[(ATA)-^1 }TAT = A[AT(AT)T}'^1 AT = A(ATA)~lAT = P. D
3.2.1 Orthogonal bases
Definition 3.2.4 A basis V = {VJ}"=1 is called orthonormal if
V^7 V. = (°^^J
(ortagonality)
— j (normalization)
Example 3.2.5 E — {ej}™=1 is an orthonormal basis for M", whereas X =
{xi}"=1 in Example 2.1.12 is not.
Proposition 3.2.6 If A is an m by n matrix whose columns are orthonormal
(called an orthogonal matrix), then ATA = In.
P = AAT = aiaj H h anaTn =4> x = ATb
is the least squared solution for Ax = b.
Corollary 3.2.7 An orthogonal matrix Q has the following properties:
- QTQ = I = QQT>
- QT = Q~\
- QT is orthogonal.
Example 3.2.8 Suppose we project a point aT = (a,b,c) into R^2 plane.
Clearly, p — (a, b, 0) as it can be seen in Figure 3.4-
T
e\ex a =
a
0
0
i e 2 e|,Q =
P = eiej + e 2 e2 =
Pa =
"100'
010
0 0( )
a
b
c
"0"
b
0
"100"
010
0 0 0_
=
a
b
0