708 E. LAGRANGE MULTIPLIERS
Figure E.1 A geometrical picture of the technique of La-
grange multipliers in which we seek to maximize a
functionf(x), subject to the constraintg(x)=0.
IfxisDdimensional, the constraintg(x)=0cor-
responds to a subspace of dimensionalityD− 1 ,
indicated by the red curve. The problem can
be solved by optimizing the Lagrangian function
L(x,λ)=f(x)+λg(x).
∇f(x)
∇g(x)
xA
g(x)=0
then parallel to the constraint surfaceg(x)=0, we see that the vector∇gis normal
to the surface.
Next we seek a pointxon the constraint surface such thatf(x)is maximized.
Such a point must have the property that the vector∇f(x)is also orthogonal to the
constraint surface, as illustrated in Figure E.1, because otherwise we could increase
the value off(x)by moving a short distance along the constraint surface. Thus∇f
and∇gare parallel (or anti-parallel) vectors, and so there must exist a parameterλ
such that
∇f+λ∇g=0 (E.3)
whereλ=0is known as aLagrange multiplier. Note thatλcan have either sign.
At this point, it is convenient to introduce theLagrangianfunction defined by
L(x,λ)≡f(x)+λg(x). (E.4)
The constrained stationarity condition (E.3) is obtained by setting∇xL=0. Fur-
thermore, the condition∂L/∂λ=0leads to the constraint equationg(x)=0.
Thus to find the maximum of a functionf(x)subject to the constraintg(x)=0,
we define the Lagrangian function given by (E.4) and we then find the stationary
point ofL(x,λ)with respect to bothxandλ. For aD-dimensional vectorx, this
givesD+1equations that determine both the stationary pointxand the value ofλ.
If we are only interested inx, then we can eliminateλfrom the stationarity equa-
tions without needing to find its value (hence the term ‘undetermined multiplier’).
As a simple example, suppose we wish to find the stationary point of the function
f(x 1 ,x 2 )=1−x^21 −x^22 subject to the constraintg(x 1 ,x 2 )=x 1 +x 2 −1=0,as
illustrated in Figure E.2. The corresponding Lagrangian function is given by
L(x,λ)=1−x^21 −x^22 +λ(x 1 +x 2 −1). (E.5)
The conditions for this Lagrangian to be stationary with respect tox 1 ,x 2 , andλgive
the following coupled equations:
− 2 x 1 +λ =0 (E.6)
− 2 x 2 +λ =0 (E.7)
x 1 +x 2 −1=0. (E.8)
