Engineering Optimization: Theory and Practice, Fourth Edition

94 Classical Optimization Techniques

(necessary conditions):

∂L

∂xi

(X,Y,λ)=

∂f

∂xi

(X)+

∑m

j= 1

λj

∂gj

∂xi

(X)= 0 , i= 1 , 2 ,... , n (2.62)

∂L

∂λj

(X,Y,λ)=Gj(X,Y)=gj(X)+y^2 j= 0 , j= 1 , 2 ,... , m (2.63)

∂L

∂yj

(X,Y,λ)= 2 λjyj= 0 , j= 1 , 2 ,... , m (2.64)

It can be seen that Eqs. (2.62) to (2.64) represent(n+ 2 m)equations in the(n+ 2 m)

unknowns,X,λ, andY. The solution of Eqs. (2.62) to (2.64) thus gives the optimum

solution vector, X∗; the Lagrange multiplier vector, λ∗; and the slack variable

vector,Y∗.

Equations(2.63) ensure that the constraintsgj( X)≤ 0 ,j= 1 , 2 ,... , m, are satis-

fied, while Eqs. (2.64) imply that eitherλj= or 0 yj=. If 0 λj= , it means that the 0

jth constraint is inactive†and hence can be ignored. On the other hand, ifyj= , it 0

means that the constraint is active (gj= ) at the optimum point. Consider the division 0

of the constraints into two subsets,J 1 andJ 2 , whereJ 1 +J 2 represent the total set of

constraints. Let the setJ 1 indicate the indices of those constraints that are active at the

optimum point andJ 2 include the indices of all the inactive constraints.

Thus forj∈J 1 ,‡yj= (constraints are active), for 0 j∈J 2 ,λj= (constraints 0

are inactive), and Eqs. (2.62) can be simplified as

∂f

∂xi

+

∑

j∈J 1

λj

∂gj

∂xi

= 0 , i= 1 , 2 ,... , n (2.65)

Similarly, Eqs. (2.63) can be written as

gj(X)= 0 , j∈J 1 (2.66)

gj(X)+y^2 j= 0 , j∈J 2 (2.67)

Equations (2.65) to (2.67) representn+p+(m−p)=n+mequations in then+m

unknownsxi(i = 1 , 2 ,... , n),λj(j∈J 1 ) and, yj(j∈J 2 ) where, pdenotes the number

of active constraints.

Assuming that the firstpconstraints are active, Eqs. (2.65) can be expressed as

−

∂f

∂xi

=λ 1

∂g 1

∂xi

+λ 2

∂g 2

∂xi

+ .. .+λp

∂gp

∂xi

, i= 1 , 2 ,... , n (2.68)

These equations can be written collectively as

−∇f=λ 1 ∇g 1 +λ 2 ∇g 2 + · · · +λp∇gp (2.69)

†Those constraints that are satisfied with an equality sign,gj= , at the optimum point are called the 0

active constraints, while those that are satisfied with a strict inequality sign,gj< , are termed 0 inactive

constraints.

‡The symbol∈is used to denote the meaning “belongs to” or “element of ”.