Engineering Optimization: Theory and Practice, Fourth Edition

64 Classical Optimization Techniques

Figure 2.1 Relative and global minima.

exists as a definite number, which we want to prove to be zero. Sincex∗is a relative

minimum, we have

f (x∗) ≤f(x∗+ h)

for all values ofhsufficiently close to zero. Hence

f (x∗+ h)−f (x∗)

h

≥0 ifh> 0

f (x∗+ h)−f (x∗)

h

≤0 ifh < 0

Thus Eq. (2.1) gives the limit ashtends to zero through positive values as

f′(x∗)≥ 0 (2.2)

while it gives the limit ashtends to zero through negative values as

f′(x∗)≤ 0 (2.3)

The only way to satisfy both Eqs. (2.2) and (2.3) is to have

f′(x∗)= 0 (2.4)

This proves the theorem.

Notes:

1.This theorem can be proved even ifx∗is a relative maximum.

2 .The theorem does not say what happens if a minimum or maximum occurs at

a pointx∗where the derivative fails to exist. For example, in Fig. 2.2,

lim

h→ 0

f (x∗+ h)−f (x∗)

h

=m+( ositivep )orm−( egativen )

depending on whetherhapproaches zero through positive or negative values,

respectively. Unless the numbersm+andm−are equal, the derivativef′(x∗)

does not exist. Iff′(x∗) does not exist, the theorem is not applicable.