Mathematical Methods for Physics and Engineering : A Comprehensive Guide

MATRICES AND VECTOR SPACES

the number of simultaneous equationsMis equal to the number of unknownsN.

This technique is known assingular value decomposition(SVD) and is the method

of choice in analysinganyset of simultaneous linear equations.

We will consider the general case, in whichAis anM×N(complex) matrix.

Let us suppose we can writeAas the product§

A=USV†, (8.131)

where the matricesU,SandVhave the following properties.

(i) The square matrixUhas dimensionsM×Mand isunitary.

(ii) The matrixShas dimensionsM×N(thesamedimensionsasthoseofA)

and isdiagonalin the sense thatSij=0ifi=j. We denote its diagonal

elements bysifori=1, 2 ,...,p,wherep= min(M, N); these elements are

termed thesingular valuesofA.

(iii) The square matrixVhas dimensionsN×Nand isunitary.

We must now determine the elements of these matrices in terms of the elements of

A. From the matrixA, we can construct two square matrices:A†Awith dimensions

N×NandAA†with dimensionsM×M. Both are clearlyHermitian. From (8.131),

and using the fact thatUandVare unitary, we find

A†A=VS†U†USV†=VS†SV† (8.132)

AA†=USV†VS†U†=USS†U†, (8.133)

whereS†SandSS†are diagonal matrices with dimensionsN×NandM×M

respectively. The firstpelements of each diagonal matrix ares^2 i,i=1, 2 ,...,p,

wherep= min(M, N), and the rest (where they exist) are zero.

These two equations imply that bothV−^1 A†AV

(

=V−^1 A†A(V†)−^1

)

and, by

a similar argument,U−^1 AA†U, must be diagonal. From our discussion of the

diagonalisation of Hermitian matrices in section 8.16, we see that the columns of

Vmust therefore be the normalised eigenvectorsvi,i=1, 2 ,...,N, of the matrix

A†Aand the columns ofUmust be the normalised eigenvectorsuj,j=1, 2 ,...,M,

of the matrixAA†. Moreover, the singular valuessimust satisfys^2 i=λi,where

theλiare the eigenvalues of the smaller ofA†AandAA†. Clearly, theλiare

also some of the eigenvalues of the larger of these two matrices, the remaining

ones being equal to zero. Since each matrix is Hermitian, theλiare real and the

singular valuessimay be taken as real and non-negative. Finally, to make the

decomposition (8.131) unique, it is customary to arrange the singular values in

decreasing order of their values, so thats 1 ≥s 2 ≥···≥sp.

§The proof that such a decomposition always exists is beyond the scope of this book. For a

full account of SVD one might consult, for example, G. H. Golub and C. F. Van Loan,Matrix

Computations, 3rd edn (Baltimore MD: Johns Hopkins University Press, 1996).