Engineering Optimization: Theory and Practice, Fourth Edition

84 Classical Optimization Techniques

As an example, consider the problem of minimizing

f (X)=f (x 1 , x 2 , x 3 )

subject to the only constraint

g 1 (X)=x 12 +x^22 +x 32 − 8 = 0

Sincen=3 andm=1 in this problem, one can think of any of themvariables,

sayx 1 , to be dependent and the remainingn−mvariables, namelyx 2 andx 3 , to be

independent.Here the constrained partial derivative (∂f/∂x 2 )g, for example, means

the rate of change offwith respect tox 2 (holding the other independent variablex 3

constant) and at the same time allowingx 1 to change aboutX∗so as to satisfy the

constraintg 1 ( X)= 0. In the present case, this means thatdx 1 has to be chosen to

satisfy the relation

g 1 (X∗+dX)≃g 1 (X∗)+

∂g 1

∂x 1

(X∗)dx 1 +

∂g 1

∂x 2

(X∗)dx 2 +

∂g 1

∂x 3

(X∗)dx 3 = 0

thatis,

2 x 1 ∗dx 1 + 2 x∗ 2 dx 2 = 0

sinceg 1 (X∗) = 0 at the optimum point anddx 3 = 0 (x 3 is held constant).

Notice that(∂f/∂xi)ghas to be zero fori=m+ 1 ,m+ 2 ,... , nsince thedxi

appearing in Eq. (2.28) are all independent. Thus the necessary conditions for the

existenceof constrained optimum atX∗can also be expressed as

(

∂f

∂xi

)

g

= 0 , i=m+ 1 , m+ 2 ,... , n (2.29)

Of course, with little manipulation, one can show that Eqs. (2.29) are nothing but

Eqs. (2.26). Further, as in the case of optimization of a multivariable function with no

constraints, one can see that a sufficient condition forX∗to be a constrained relative

minimum (maximum) is that the quadratic formQdefined by

Q=

∑n

i=m+ 1

∑n

j=m+ 1

(

∂^2 f

∂xi∂xj

)

g

dxidxj (2.30)

is positive (negative) for all nonvanishing variationsdxi. As in Theorem 2.4, the matrix

          

(

∂^2 f

∂x^2 m+ 1

)

g

(

∂^2 f

∂xm+ 1 ∂xm+ 2

)

g

· · ·

(

∂^2 f

∂xm+ 1 ∂xn

)

g

..

.

(

∂^2 f

∂xn∂xm+ 1

)

g

(

∂^2 f

∂xn∂xm+ 2

)

g

· · ·

(

∂^2 f

∂x^2 n

)

g

          

has to be positive (negative) definite to haveQpositive (negative) for all choices of

dxi. It is evident that computation of the constrained derivatives (∂^2 f/∂xi∂xj)gis a