Engineering Optimization: Theory and Practice, Fourth Edition

2.3 Multivariable Optimization with No Constraints 71

will have the same sign as (∂^2 f/∂xi∂xj)|X=X∗for all sufficiently smallh.Thus
f (X∗+ h)−f(X∗) willbe positive, and henceX∗will be a relative minimum, if

Q=

∑n

i= 1

∑n

j= 1

hihj

∂^2 f

∂xi∂xj

∣

∣

∣

∣

X=X∗

(2.11)

is positive. This quantityQis a quadratic form and can be written in matrix form as

Q=hTJh|X=X∗ (2.12)

where

J|X=X∗=

[

∂^2 f

∂xi∂xj

∣

∣

∣

∣

X=X∗

]

(2.13)

is the matrix of second partial derivatives and is called theHessian matrixoff(X).
It is known from matrix algebra that the quadratic form of Eq. (2.11) or (2.12)
will be positive for allhif and only if [J] is positive definite atX=X∗. This means
that a sufficient condition for the stationary pointX∗to be a relative minimum is that
the Hessian matrix evaluated at the same point be positive definite. This completes the
proof for the minimization case. By proceeding in a similar manner, it can be proved
that the Hessian matrix will be negative definite ifX∗is a relative maximum point.
Note:A matrixAwill be positive definite if all its eigenvalues are positive; that
is, all the values ofλthat satisfy the determinantal equation

|A−λI| = 0 (2.14)

should be positive. Similarly, the matrix [A] will be negative definite if its eigenvalues
are negative.
Another test that can be used to find the positive definiteness of a matrixAof
orderninvolves evaluation of the determinants

A= |a 11 |,

A 2 =

∣

∣

∣

∣

a 11 a 12

a 21 a 22

∣

∣

∣

∣,

A 3 =

∣ ∣ ∣ ∣ ∣ ∣

a 11 a 12 a 13

a 21 a 22 a 23

a 31 a 32 a 32

∣ ∣ ∣ ∣ ∣ ∣

,... ,

An=

∣ ∣ ∣ ∣ ∣ ∣ ∣ ∣ ∣ ∣ ∣

a 11 a 12 a 13 · · · a 1 n

a 21 a 22 a 23 · · · a 2 n

a 31 a 32 a 33 · · · a 3 n

..

.

an 1 an 2 an 3 · · · ann

∣ ∣ ∣ ∣ ∣ ∣ ∣ ∣ ∣ ∣ ∣

(2.15)

The matrixAwill be positive definite if and only if all the valuesA 1 , A 2 , A 3 ,... , An
are positive. The matrixAwill be negative definite if and only if the sign ofAjis
(–1)j forj= 1 , 2 ,... , n. If some of theAj are positive and the remainingAj are
zero, the matrixAwill be positive semidefinite.

Example 2.4 Figure 2.4 shows two frictionless rigid bodies (carts)AandBconnected
by three linear elastic springs having spring constantsk 1 , k 2 , andk 3. The springs are
at their natural positions when the applied forcePis zero. Find the displacementsx 1
andx 2 under the forcePby using the principle of minimum potential energy.