Mathematical Tools for Physics

11—Numerical Analysis 341

Minimize D^2 =

∑

wi′^2 −

∑

(~w′i.~v)^2 subject to φ=vx^2 +vy^2 −1 = 0

The independent variables arevxandvy, and the problem becomes

∇

(

D^2 +λφ

)

= 0, with φ= 0

Do the differentiations with respect to the independent variables and you have two linear equations forvxandvy,

−

∂

∂vx

∑(

wxi′vx+wyi′vy

) 2

+λ 2 vx= 0 or

−

∑

2

(

w′xivx+w′yivy

)

wxi+λ 2 vx= 0

−

∑

2

(

w′xivx+w′yivy

)

wyi+λ 2 vy= 0

(53)

Correlation, Principal Components
The correlation matrix of this data is

(C) =

1

N

( ∑

w′xi^2

∑

wxiw′yi

∑

wyi′wxi′

∑

wyi′^2

)

The equations ( 53 ) are (
Cxx Cxy
Cyx Cyy

)(

vx

vy

)

=λ′

(

vx

vy

)

(54)

whereλ′=λ/N. This is a traditional eigenvector equation, and there is a non-zero solution only if the determinant
of the coefficients equals zero. Which eigenvalue do I pick? There are two of them, and one will give the best fit
while the other gives theworstfit. Just because the first derivative is zero doesn’t mean you have a minimum
ofD^2 ; it could be a maximum. Here the answer is that you pick the larger eigenvalue. You can see why this is
plausible by looking at the special case for which all the data lie along thex-axis, thenCxx> 0 and all the other
components of the matrix= 0. The eigenvalues areCxxand zero, and the corresponding eigenvectors areˆxand
ˆyrespectively. Clearly the best fit corresponds to the former, and the best fit line is thex-axis. The general form
of the best fit line is (now using the original coordinate system for the data)

αˆv+

1

N

∑

~wi=αˆv+~wmean