OPTIMIZATION 351
whereλi are called the Lagrangian multipliers. The Lagrangian L(X,ΛΛΛΛΛ) is treated as an unconstrained
function of X and ΛΛΛΛΛ to be minimized. Following the derivation in section 12.3.1, the necessary
conditions for L(X,ΛΛΛΛΛ) to have an extremum at (X 0 ,ΛΛΛΛΛ 0 ) are that the first partial derivatives of
L(X,ΛΛΛΛΛ) with respect to the n + m variables are all zero, that is
∂
∂
∂
∂
∂
∂
L(, )
=
()
+
()
(^0) =
=1
X 0
X
X
X
X
X
ΛΛ fg
i
m
i
Σλ i 0
and
∂
∂
L(, )
= ( 0 ) = 0, = 1,... ,
X
X
ΛΛ
ΛΛ
gimi (12.14)
The above are n + m equations which can be solved for the same number of variables. The sufficiency
condition for L(X,ΛΛΛΛΛ) to have a relative minimum at X 0 is that the Hessian matrix Hpq = ∂^2 L(X 0 ,ΛΛΛΛΛ 0 )/
∂xp∂xq,p,q = 1,... , n should be positive definite at X = X 0 for values of ΔX for which all the
constraints are satisfied.
The above conditions may be derived in a manner similar to that in the unconstrained case. Let
G(X)≡ [g 1 (X),g 2 (X),... , gm(X)] so that L(X,ΛΛΛΛΛ) = f(X) + G(X)ΛΛΛΛΛ. Consider the Taylor’s expansion
of the augmented Lagrangian up to the first derivatives, that is
LLL(XX 00 + , + ) = (X 00 , ) + L L +
X
ΔΔΛΛΛΛ ΔX Δ
ΛΛ
∂ ΛΛ
∂
⎡
⎣⎢
⎤
⎦⎥
∂
∂
⎡
⎣⎢
⎤
⎦⎥
or
LL(XX 00 + , + ) – (X 00 , ) = ( 0 ) + ( 00 ) + ( 0 )
X
X
X
ΔΔΛΛΛΛΛΛΛ∂ GX ΛΛΔX GXΔΛ
∂
∂
∂
⎧
⎨
⎩
⎫
⎬
⎭
⎡
⎣
⎢
⎤
⎦
f ⎥
(12.15)
ForL(X 0 + ΔX,ΛΛΛΛΛ 0 + ΔΛΛΛΛΛ)≥L(X 0 ,ΛΛΛΛΛ 0 ) at a local minimum, we have
∂
∂
∂
∂
⎧
⎨
⎩
⎫
⎬
⎭
⎡
⎣
⎢
⎤
⎦
X () + f X^0000 XGX()ΛΛΛ ⎥ΔΔX GX + () 0Λ≥ (12.16)
for all small variations ΔX and ΔΛΛΛΛΛ which is only possible if
∂
∂
∂
∂
⎧
⎨
⎩
⎫
⎬
X ⎭
X
X
() + f 000 GX() = ΛΛ0 0
and G(X 0 ) = 0 ,i = 1,... , m
which are the conditions stated in Eq. (12.14). Considering the expansion of the Lagrangian to
include the second derivatives and noting that the coefficients of ΔX and ΔΛΛΛΛΛ are both 0 from the
necessary condition, we have
L(X 0 + ΔX,ΛΛΛΛΛ 0 + ΔΛΛΛΛΛ) – L(X 0 ,ΛΛΛΛΛ 0 )
=^12 T (, ) +^12 (, ) + (, )
2
2 00
2
2 00
2
ΔΔΔΔΔ ΔX X XX X X XX 00
∂
∂
∂
∂
∂
∂∂
LLLΛΛΛΛ
ΛΛ
ΛΛΛΛΛΛ
ΛΛ
TTΛΛ
SinceL(X,ΛΛΛΛΛ) is linear in ΛΛΛΛΛ, ∂
∂
2
ΛΛ^2 L(X^0 ,ΛΛ
ΛΛΛ 0 ) = 0. Hence