Dynamic programming
DeÖnition
A strategy is a Markovian strategy if the strategy is dependent only on the
present state, that is independent of the history of path how we have got
to the present states.Theorem (Bellmanís principle)
Under some conditions on the model there is an optimal Markovian
strategy and the sequence of value functions VtsatisÖes the Bellman
equation
Vt(s)= max
a 2 Φt(s)frt(s,a)+Vt+ 1 (ft(s,a))g.