Engineering Optimization: Theory and Practice, Fourth Edition

338 Nonlinear Programming II: Unconstrained Optimization Techniques

This formula requires two additional function evaluations for each of the partial deriva-

tives. In Eqs. (6.63) and (6.64),

xiis a small scalar quantity anduiis a vector of order

nwhoseith component has a value of 1, and all other components have a value of zero.

In practical computations, the value of

xihas to be chosen with some care. If xiis

too small, the difference between the values of the function evaluated at(Xm+ xiui)

and(Xm− xiui) ay be very small and numerical round-off error may predominate.m

On the other hand, if

xiis too large, the truncation error may predominate in the

calculation of the gradient.

In the second case also, the use of finite-difference formulas is preferred whenever

the exact gradient evaluation requires more computational time than the one involved

in using Eq. (6.63) or (6.64).

In the third case, we cannot use the finite-difference formulas since the gradient

is not defined at all the points. For example, consider the function shown in Fig. 6.15.

If Eq. (6.64) is used to evaluate the derivativedf/dsatXm, we obtain a value of

α 1 for a step size x 1 and a value ofα 2 for a step size x 2. Since, in reality, the

derivative does not exist at the pointXm, use of finite-difference formulas might lead

to a complete breakdown of the minimization process. In such cases the minimization

can be done only by one of the direct search techniques discussed earlier.

6.8.2 Rate of Change of a Function along a Direction

In most optimization techniques, we are interested in finding the rate of change of a

function with respect to a parameterλalong a specified direction,Si, away from a

pointXi. Any point in the specified direction away from the given pointXi can be

expressed asX=Xi+λSi. Our interest is to find the rate of change of the function

along the directionSi(characterized by the parameterλ), that is,

df

dλ

=

∑n

j= 1

∂f

∂xj

∂xj

∂λ

(6.65)

a (^2) a
1
Figure 6.15 Gradientnot defined atxm.