Engineering Optimization: Theory and Practice, Fourth Edition

5.12 Direct Root Methods 287

Thus theNewton method, Eq. (5.65), is equivalent to using a quadratic approximation
for the functionf (λ)and applying the necessary conditions. The iterative process given
by Eq. (5.65) can be assumed to have converged when the derivative,f′(λi+ 1 ) is close,
to zero:

|f′(λi+ 1 ) ≤| ε (5.66)

whereεis a small quantity. The convergence process of the method is shown graphi-
cally in Fig. 5.18a.

Remarks:

1.The Newton method was originally developed by Newton for solving nonlinear

equations and later refined by Raphson, and hence the method is also known as

Newton–Raphson methodin the literature of numerical analysis.

2.The method requires both the first- and second-order derivatives off (λ).

3.Iff′′(λi) 

= 0 [in Eq. (5.65)], the Newton iterative method has a powerful

(fastest) convergence property, known asquadratic convergence.†

4. f the starting point for the iterative process is not close to the true solutionI λ∗,
   the Newton iterative process might diverge as illustrated in Fig. 5.18b.

li

li

l*

l*

f′(l)

l

f′(l)

l

Tangent at li

o

(a)

(b)

o

Tangent at li

Tangent at li+ 1

Tangent at li+ 1

li+ 2

li+ 2

li+ 1

li+ 1

Figure 5.18 Iterative process of Newton method:(a)convergence;(b)divergence.

†The definition of quadratic convergence is given in Section 6.7.