MATRICES AND VECTOR SPACES
Using the unitarity of the matricesUandV, we find that
Axˆ−b=USSU†b−b=U(SS−I)U†b. (8.142)The matrix (SS−I) is diagonal and thejth element on its leading diagonal is
non-zero (and equal to−1) only whensj= 0. However, thejth element of the
vectorU†bis given by the scalar product (uj)†b;ifblies in the range ofA,this
scalar product can be non-zero only ifsj= 0. Thus the RHS of (8.142) must
equal zero, and soˆxgiven by (8.141) is a solution to the equationsAx=b.We
may, however, add to this solutionanylinear combination of theN−rvectorsvi,
i=r+1,r+2,...,N, that form an orthonormal basis for the null space ofA; thus,
in general, there exists an infinity of solutions (although it is straightforward to
show that (8.141) is the solution vector of shortest length). The only way in which
the solution (8.141) can beuniqueis if the rankrequalsN, so that the matrixA
does not possess a null space; this only occurs ifAis square and non-singular.
Ifbdoes not lie in the range ofAthen the set of equationsAx=bdoesnot have a solution. Nevertheless, the vector (8.141) provides the closest possible
‘solution’ in a least-squares sense. In other words, although the vector (8.141)
does not exactly solveAx=b, it is the vector that minimises theresidual
=|Ax−b|,where here the vertical lines denote the absolute value of the quantity they
contain, not the determinant. This is proved as follows.
Suppose we were to add some arbitrary vectorx′to the vectorxˆin (8.141).This would result in the addition of the vectorb′=Ax′toAxˆ−b;b′is clearly in
the range ofAsince any part ofx′belonging to the null space ofAcontributes
nothing toAx′. We would then have
|Aˆx−b+b′|=|(USSU†−I)b+b′|=|U[(SS−I)U†b+U†b′]|=|(SS−I)U†b+U†b′|; (8.143)in the last line we have made use of the fact that the length of a vector is left
unchanged by the action of the unitary matrixU. Now, thejth component of the
vector (SS−I)U†bwill only be non-zero whensj= 0. However, thejth element
of the vectorU†b′is given by the scalar product (uj)†b′, which is non-zero only if
sj=0,sinceb′lies in the range ofA. Thus, as these two terms only contribute to
(8.143) for two disjoint sets ofj-values, its minimum value, asx′is varied, occurs
whenb′= 0 ; this requiresx′= 0.
Find the solution(s) to the set of simultaneous linear equationsAx=b,whereAis given
by (8.137) andb=(100)T.To solve the set of equations, we begin by calculating the vector given in (8.141),
x=VSU†b,