Mathematical Modeling in Finance with Stochastic Processes

1.7. STOCHASTIC PROCESSES 59

The defining property of a Markov chain is that

P[Xj=l|X 0 =k 0 ,X 1 =k 1 ,...,Xj− 1 =kj− 1 ] =P[Xj=l|Xj− 1 =kj− 1 ].

In words, the future is conditionally independent of the past, the probability
of transition from statekj− 1 at timej−1 to statelat timejdepends only
onkj− 1 andl, not on the historyX 0 =k 0 ,X 1 =k 1 ,...,Xj− 2 =kj− 2 of how
the process got tokj− 1
A simple random walk is an example of a Markov chain. The states are
the integers and the transition probabilities are

P[Xj=l|Xj− 1 =k] = 1/2 ifl=k−1 orl=k+ 1

P[Xj=l|Xj− 1 =k] = 0 otherwise

Another example would be the position of a game piece in the board
game Monopoly. The index set is the counting numbers listing the plays
of the game. The sample space is the set of infinite sequence of rolls of a
pair of dice. The state space is the set of 40 real-estate properties and other
positions around the board.
Markov chains have been extended to making optimal decisions under
uncertainty as “Markov decision processes”. Another extension to signal pro-
cessing and bioinformatics is called “hidden Markov models”. Markov chains
are an important and useful class of stochastic processes. Mathematicians
have extensively studied and classified Markov chains and their extensions
but we will not have reason to examine them carefully in this text.
A generalization of a Markov chain is aMarkov process. In a Markov
process, we allow the index set to be either a discrete set of times as the
integers or an interval, such as the non-negative reals. Likewise the state
space may be either a set of discrete values or an interval, even the whole
real line. In mathematical notation a stochastic processX(t) is calledMarkov
if for everynandt 1 < t 2 < ... < tnand real numberxn, we have

P[X(tn)≤xn|X(tn− 1 ),...,X(t 1 )] =P[X(tn)≤xn|X(tn− 1 )].

Many of the models we use in this text will naturally be taken as Markov
processes because of the intuitive appeal of this “memory-less” property.
Many stochastic processes are naturally expressed as taking place in a
discrete state space with a continuous time index. For example, consider ra-
dioactive decay, counting the number of atomic decays which have occurred