9.9 Continuous Dynamic Programming 573
These comments are equally applicable for all dynamic programming problems
involving many state variables, since the computations have to be performed for dif-
ferent possible values of each of the state variables. Thus this problem causes not only
an increase in the computational time, but also requires a large computer memory. This
problem is known as theproblem of dimensionalityor thecurse of dimensionality, as
termed by Bellman. This presents a serious obstacle in solving medium- and large-size
dynamic programming problems.
9.9 Continuous Dynamic Programming
If the number of stages in a multistage decision problem tends to infinity, the problem
becomes an infinite stage or continuous problem and dynamic programming can still
be used to solve the problem. According to this notion, the trajectory optimization
problems, defined in Section 1.5, can also be considered asinfinite-stageorcontinuous
problems.
An infinite-stage or continuous decision problem may arise in several practical
problems. For example, consider the problem of a missile hitting a target in a specified
(finite) time interval. Theoretically, the target has to be observed and commands to the
missile for changing its direction and speed have to be given continuously. Thus an
infinite number of decisions have to be made in a finite time interval. Since a stage has
been defined as a point where decisions are made, this problem will be an infinite-stage
or continuous problem. Another example where an infinite-stage or continuous decision
problem arises is in planning problems. Since large industries are assumed to function
for an indefinite amount of time, they have to do their planning on this basis. They
make their decisions at discrete points in time by anticipating a maximum profit in the
long run (essentially over an infinite period of time). In this section we consider the
application of continuous decision problems.
We have seen that the objective function in dynamic programming formulation
is given by the sum of individual stage returns. If the number of stages tends
to infinity, the objective function will be given by the sum of infinite terms,
which amounts to having the objective function in the form of an integral. The
following examples illustrate the formulation of continuous dynamic programming
problems.
Example 9.6 Consider a manufacturing firm that produces a certain product. The rate
of demand of this product (p) is known to bep=p[x(t), t], wheretis the time of
the year andx(t)is the amount of money spent on advertisement at timet. Assume
that the rate of production is exactly equal to the rate of demand. The production cost,
c, is known to be a function of the amount of production (p)and the production rate
(dp/dt)asc=c(p, dp/dt). The problem is to find the advertisement strategy,x(t),
so as to maximize the profit betweent 1 andt 2. The unit selling price (s)of the product
is known to be a function of the amount of production ass=s(p)=a+b/p, where
aandbare known positive constants.
SOLUTION Since the profit is given by the difference between the income from sales
and the expenditure incurred for production and advertisement, the total profit over the
