Computer Aided Engineering Design

OPTIMIZATION 349

f(X 0 + ΔX)≤f (X 0 ) for a relative maximum at X 0 as well for which reason X 0 is called a stationary
point. The condition is analogous to the one variable case where the slopes at the extrema are zero.
Considering the series expansion up to the second term, we have

ff

ffT

( + ) = ( ) +

()

+^1

2

()

00

0 2 0

XXX 2

X

X

XX

X

X

ΔΔΔΔX

∂

∂

∂

∂

(12.11)

At an extremum, after applying the necessary conditions, the above equation becomes

ff

T f

( + ) – ( ) =^1

2

()

00

2

0

XXX X 2

X

X

ΔΔΔX

∂

∂

(12.12)

ForX 0 to be a relative minimum, we require from above that^1
2

()

> 0.

2

0

ΔΔX 2

X

X

T X

∂ f

∂

Note that

∂

∂

2

0

2

f()X

X

is a square matrix Hij of dimension n×n written such that Hij = ∂^2 f(X 0 )/∂xi∂xj,i,j=

1,... n, and is known as the Hessianof the function f(X). For ΔXTH(X 0 )ΔX > 0 for any ΔX, we
require that H(X 0 ) be positive definite by definition^2. For X 0 to be a relative maximum, using similar
arguments, we require that ΔXTH(X 0 )ΔX < 0 for any ΔX or H(X 0 ) to be negative definite. A way to
determine the definiteness of a matrix is by its eignvalues λ such that AX = λX and so det(A – λI)
= 0. For all positive (and non-zero) eigen values, the matrix is positive definite whereas for all
negative eignvalues, the latter is negative definite. A stationary point X 0 is said to be a saddle point
if the Hessian at X 0 is neither positive nor negative definite.

Example 12.6. Consider the function f (x,y) = x^2 + y^2 (Figure 12.8 a). The function is bowl-shaped
and has a single global extremum, a minimum. Note that there is a stationary point at x = 0, y= 0
which can be obtained using ∂f/∂x = 2x = 0 and ∂f/∂y = 2y= 0. Also, note that ∂^2 f/∂x^2 = ∂^2 f/∂y^2 =
2, whereas ∂^2 f/∂x∂y = 0 that makes the Hessian H = 2I 2 × 2 which is positive definite. Next, consider the
plot of f(x,y) = (y^3 – 3y) (1 + x^2 )–1 in the region x∈ [–2, 2] and y∈[–2, 2] (Figure 12.8b). The partial
derivatives can be computed as ∂f/∂x= – 2x(y^3 – 3y)(1 + x^2 )–2 and ∂f/∂y = (3y^2 – 3) (1 + x^2 )–1, and
setting both to zero yields x= 0 and y=± 1. Notice the local maximum at (0, –1) and local minimum
at (0, 1). Further, ∂^2 f/∂x^2 = –2(y^3 – 3y) [(1 + x^2 )–2 – 4x^2 (1 + x^2 )–3],∂^2 f/∂y^2 = (6y) (1 + x^2 )–1and
∂^2 f/∂x∂y = –2x(3y^2 – 3) (1 + x^2 )–2. At (0, –1), ∂^2 f/∂x^2 = –4, ∂^2 f/∂y^2 = –6 and ∂^2 f/∂x∂y = 0 for which
the eigenvalues of the resulting Hessian are –4 and –6 (both negative) which confirms the local
maximum. At (0, 1), ∂^2 f/∂x^2 = 4, ∂^2 f/∂y^2 = 6 and ∂^2 f/∂x∂y= 0 for which the eigen values are
positive and the point is a relative minimum.
Forf(x,y) = 4 – (x^2 + y^2 ) (Figure 12.8 c), the function is bowl-shaped and has a single global
maximum. Note the stationary point at x = 0, y= 0 for which it is straightforward to show that the
Hessian is negative definite. Finally, consider f(x,y) = x^2 – y^2 (Figure 12.8 d). The function has a
single saddle point (neither a minimum nor a maximum). At the stationary point (0, 0), ∂^2 f/∂x^2 = 2,
∂^2 f/∂y^2 = –2 and ∂^2 f/∂x∂y= 0 for which the Hessian has one positive and one negative eigenvalue.

12.3.2 Constrained Multivariable Optimization

For engineering design problems, the design goal is almost always associated with some constraints.

(^2) A matrix A is positive definite if for all X = {x
1 ,x 2 ,... , xn}
T,XTAX > 0 and XTAX= 0 iff X = 0. A negative
definiteB satisfies the relation XTBX < 0 for all X and XTBX = 0 only for X = 0.