Engineering Optimization: Theory and Practice, Fourth Edition

68 Classical Optimization Techniques

2.3 Multivariable Optimization with No Constraints

In this section we consider the necessary and sufficient conditions for the minimum

or maximum of an unconstrained function of several variables. Before seeing these

conditions, we consider the Taylor’s series expansion of a multivariable function.

Definition:rth Differential off. If all partial derivatives of the functionfthrough

orderr≥1 exist and are continuous at a pointX∗, the polynomial

drf (X∗)=

∑n

i= 1

∑n

j= 1

· · ·

∑n

k= 1

︸ ︷︷ ︸

r ummationss

hihj· · ·hk

∂rf (X∗)

∂xi∂xj· · · ∂xk

(2.6)

is called therth differential off atX∗. Notice that there arersummations and onehi

is associated with each summation in Eq. (2.6).

For example, whenr=2 andn=3, we have

d^2 f (X∗)=d^2 f (x 1 ∗, x∗ 2 , x∗ 3 )=

∑^3

i= 1

∑^3

j= 1

hihj

∂^2 f (X∗)

∂xi∂xj

=h^21

∂^2 f

∂x^21

(X∗)+h^22

∂^2 f

∂x^22

(X∗)+h^23

∂^2 f

∂x^23

(X∗)

+ 2 h 1 h 2

∂^2 f

∂x 1 ∂x 2

(X∗)+ 2 h 2 h 3

∂^2 f

∂x 2 ∂x 3

(X∗)+ 2 h 1 h 3

∂^2 f

∂x 1 ∂x 3

(X∗)

The Taylor’s series expansion of a functionf(X) about a pointX∗is given by

f(X)=f (X∗) +df (X∗)+

1

2!

d^2 f (X∗)+

1

3!

d^3 f (X∗)

+· · · +

1

N!

dNf (X∗)+RN(X∗,h) (2.7)

where the last term, called theremainder, is given by

RN(X∗,h)=

1

(N+ 1 )!

dN+^1 f (X∗+θh) (2.8)

where 0< θ <1 andh=X−X∗.

Example 2.3 Find the second-order Taylor’s series approximation of the function

f (x 1 , x 2 , x 3 )=x^22 x 3 +x 1 ex^3

about the pointX∗= { 1 , 0 ,− 2 }T.

SOLUTION The second-order Taylor’s series approximation of the functionfabout

pointX∗is given by

f(X)=f





1

0

− 2



+df





1

0

− 2



+^1

2!

d^2 f





1

0

− 2



