Engineering Optimization: Theory and Practice, Fourth Edition

306 Nonlinear Programming II: Unconstrained Optimization Techniques

design variables changes the condition number†of the Hessian matrix. When the con-

dition number of the Hessian matrix is 1, the steepest descent method, for example,

finds the minimum of a quadratic objective function in one iteration.

Iff=^12 XT[ A]Xdenotes a quadratic term, a transformation of the form

X=[R]Y or

{

x 1

x 2

}

=

[

r 11 r 12

r 21 r 22

]{

y 1

y 2

}

(6.7)

can be used to obtain a new quadratic term as

1

2 Y

T[A ̃]Y= 1

2 Y

T[R]T[ [A]R]Y (6.8)

The matrix [R] can be selected to make [A ̃]=[R]T[ [A]R] diagonal (i.e., to eliminate

the mixed quadratic terms). For this, the columns of the matrix [R] are to be chosen

as the eigenvectors of the matrix [A]. Next the diagonal elements of the matrix [A ̃]

can be reduced to 1 (so that the condition number of the resulting matrix will be 1) by

using the transformation

Y=[S]Z or

{

y 1

y 2

}

=

[

s 11 0

0 s 22

]{

z 1

z 2

}

(6.9)

where the matrix [S] is given by

[S]=





s 11 =√^1

a ̃ 11

0

0 s 22 =√^1

a ̃ 22



 (6.10)

Thus the complete transformation that reduces the Hessian matrix off to an identity

matrix is given by

X=[R][S]Z≡[T]Z (6.11)

so that the quadratic term^12 XT[ A]Xreduces to^12 ZT[I]Z.

Ifthe objective function is not a quadratic, the Hessian matrix and hence the

transformations vary with the design vector from iteration to iteration. For example,

†The condition number of ann×nmatrix, [A], is defined as

cond([A])= ||[A]|| ||[A]−^1 || ≥ 1

where||[A]||denotes a norm of the matrix [A]. For example, the infinite norm of [A] is defined as the

maximum row sum given by

||[A]||∞= max

1 ≤i≤n

∑n

j= 1

|aij|

If the condition number is close to 1, the round-off errors are expected to be small in dealing with the

matrix [A]. For example, if cond[A] is large, the solution vectorXof the system of equations [A]X=Bis

expected to be very sensitive to small variations in [A] andB. If cond[A] is close to 1, the matrix [A] is

said to bewell behavedorwell conditioned. On the other hand, if cond[A] is significantly greater than 1,

the matrix [A] is said to benot well behavedorill conditioned.