Engineering Optimization: Theory and Practice, Fourth Edition

70 Classical Optimization Techniques

that is,

f (X∗+ h)−f(X∗)=hk

∂f

∂xk

(X∗)+

1

2!

d^2 f (X∗+ θh), 0 < θ < 1

Sinced^2 f (X∗+ θh)is of orderh^2 i, the terms of orderhwill dominate the higher-order

terms for smallh. Thus the sign off (X∗+ h)−f(X∗) s decided by the sign ofi

hk∂f (X∗)/∂xk. Suppose that∂f (X∗)/∂xk> 0. Then the sign off (X∗+ h)−f(X∗)

will be positive forhk> 0 and negative forhk<. This means that 0 X∗cannot be

an extreme point. The same conclusion can be obtained even if we assume that

∂f (X∗)/∂xk<. Since this conclusion is in contradiction with the original statement 0

thatX∗is an extreme point, we may say that∂f/∂xk= at 0 X=X∗. Hence the theorem

is proved.

Theorem 2.4 Sufficient Condition A sufficient condition for a stationary pointX∗

to be an extreme point is that the matrix of second partial derivatives (Hessian matrix)

off(X) evaluated atX∗is (i) positive definite whenX∗is a relative minimum point,

and(ii) negative definite whenX∗is a relative maximum point.

Proof: From Taylor’s theorem we can write

f (X∗+ h)=f(X∗)+

∑n

i= 1

hi

∂f

∂xi

(X∗)+

1

2!

∑n

i= 1

∑n

j= 1

hihj

∂^2 f

∂xi∂xj

∣

∣

∣

∣

X=X∗+θh

,

0 <θ < 1 (2.10)

SinceX∗is a stationary point, the necessary conditions give (Theore m 2.3)

∂f

∂xi

= 0 , i= 1 , 2 ,... , n

Thus Eq. (2.10) reduces to

f (X∗+ h)−f(X∗)=

1

2!

∑n

i= 1

∑n

j= 1

hihj

∂^2 f

∂xi∂xj

∣

∣

∣

∣

X=X∗+θh

, 0 <θ < 1

Therefore, the sign of

f (X∗+ h)−f(X∗)

will be same as that of

∑n

i= 1

∑n

j= 1

hihj

∂^2 f

∂xi∂xj

∣

∣

∣

∣

X=X∗+θh

Since the second partial derivative of∂^2 f (X)/∂xi∂xjis continuous in the neighborhood

ofX∗,

∂^2 f

∂xi∂xj

∣

∣

∣

∣

X=X∗+θh