84 Classical Optimization Techniques
As an example, consider the problem of minimizing
f (X)=f (x 1 , x 2 , x 3 )
subject to the only constraint
g 1 (X)=x 12 +x^22 +x 32 − 8 = 0
Sincen=3 andm=1 in this problem, one can think of any of themvariables,
sayx 1 , to be dependent and the remainingn−mvariables, namelyx 2 andx 3 , to be
independent.Here the constrained partial derivative (∂f/∂x 2 )g, for example, means
the rate of change offwith respect tox 2 (holding the other independent variablex 3
constant) and at the same time allowingx 1 to change aboutX∗so as to satisfy the
constraintg 1 ( X)= 0. In the present case, this means thatdx 1 has to be chosen to
satisfy the relation
g 1 (X∗+dX)≃g 1 (X∗)+
∂g 1
∂x 1
(X∗)dx 1 +
∂g 1
∂x 2
(X∗)dx 2 +
∂g 1
∂x 3
(X∗)dx 3 = 0
thatis,
2 x 1 ∗dx 1 + 2 x∗ 2 dx 2 = 0
sinceg 1 (X∗) = 0 at the optimum point anddx 3 = 0 (x 3 is held constant).
Notice that(∂f/∂xi)ghas to be zero fori=m+ 1 ,m+ 2 ,... , nsince thedxi
appearing in Eq. (2.28) are all independent. Thus the necessary conditions for the
existenceof constrained optimum atX∗can also be expressed as
(
∂f
∂xi
)
g
= 0 , i=m+ 1 , m+ 2 ,... , n (2.29)
Of course, with little manipulation, one can show that Eqs. (2.29) are nothing but
Eqs. (2.26). Further, as in the case of optimization of a multivariable function with no
constraints, one can see that a sufficient condition forX∗to be a constrained relative
minimum (maximum) is that the quadratic formQdefined by
Q=
∑n
i=m+ 1
∑n
j=m+ 1
(
∂^2 f
∂xi∂xj
)
g
dxidxj (2.30)
is positive (negative) for all nonvanishing variationsdxi. As in Theorem 2.4, the matrix
(
∂^2 f
∂x^2 m+ 1
)
g
(
∂^2 f
∂xm+ 1 ∂xm+ 2
)
g
· · ·
(
∂^2 f
∂xm+ 1 ∂xn
)
g
..
.
(
∂^2 f
∂xn∂xm+ 1
)
g
(
∂^2 f
∂xn∂xm+ 2
)
g
· · ·
(
∂^2 f
∂x^2 n
)
g
has to be positive (negative) definite to haveQpositive (negative) for all choices of
dxi. It is evident that computation of the constrained derivatives (∂^2 f/∂xi∂xj)gis a