70 Classical Optimization Techniques
that is,
f (X∗+ h)−f(X∗)=hk
∂f
∂xk
(X∗)+
1
2!
d^2 f (X∗+ θh), 0 < θ < 1
Sinced^2 f (X∗+ θh)is of orderh^2 i, the terms of orderhwill dominate the higher-order
terms for smallh. Thus the sign off (X∗+ h)−f(X∗) s decided by the sign ofi
hk∂f (X∗)/∂xk. Suppose that∂f (X∗)/∂xk> 0. Then the sign off (X∗+ h)−f(X∗)
will be positive forhk> 0 and negative forhk<. This means that 0 X∗cannot be
an extreme point. The same conclusion can be obtained even if we assume that
∂f (X∗)/∂xk<. Since this conclusion is in contradiction with the original statement 0
thatX∗is an extreme point, we may say that∂f/∂xk= at 0 X=X∗. Hence the theorem
is proved.
Theorem 2.4 Sufficient Condition A sufficient condition for a stationary pointX∗
to be an extreme point is that the matrix of second partial derivatives (Hessian matrix)
off(X) evaluated atX∗is (i) positive definite whenX∗is a relative minimum point,
and(ii) negative definite whenX∗is a relative maximum point.
Proof: From Taylor’s theorem we can write
f (X∗+ h)=f(X∗)+
∑n
i= 1
hi
∂f
∂xi
(X∗)+
1
2!
∑n
i= 1
∑n
j= 1
hihj
∂^2 f
∂xi∂xj
∣
∣
∣
∣
X=X∗+θh
,
0 <θ < 1 (2.10)
SinceX∗is a stationary point, the necessary conditions give (Theore m 2.3)
∂f
∂xi
= 0 , i= 1 , 2 ,... , n
Thus Eq. (2.10) reduces to
f (X∗+ h)−f(X∗)=
1
2!
∑n
i= 1
∑n
j= 1
hihj
∂^2 f
∂xi∂xj
∣
∣
∣
∣
X=X∗+θh
, 0 <θ < 1
Therefore, the sign of
f (X∗+ h)−f(X∗)
will be same as that of
∑n
i= 1
∑n
j= 1
hihj
∂^2 f
∂xi∂xj
∣
∣
∣
∣
X=X∗+θh
Since the second partial derivative of∂^2 f (X)/∂xi∂xjis continuous in the neighborhood
ofX∗,
∂^2 f
∂xi∂xj
∣
∣
∣
∣
X=X∗+θh