1.7. STOCHASTIC PROCESSES 57
- A generalization of a Markov chain is aMarkov Process. In a Markov
process, we allow the index set to be either a discrete set of times as
the integers or an interval, such as the non-negative reals. Likewise the
state space may be either a set of discrete values or an interval, even
the whole real line. In mathematical notation a stochastic processX(t)
is calledMarkovif for everynandt 1 < t 2 < ... < tnand real number
xn, we have
P[X(tn)≤xn|X(tn− 1 ),...,X(t 1 )] =P[X(tn)≤xn|X(tn− 1 )].Many of the models we use in this text will naturally be taken as
Markov processes because of the intuitive appeal of this “memory-less”
property.Mathematical Ideas
Definition and Notations
A sequence or interval of random outcomes, that is, random outcomes de-
pendent on time is called astochastic process. Stochastic is a synonym for
“random.” The word is of Greek origin and means “pertaining to chance”
(Greekstokhastikos, skillful in aiming; fromstokhasts, diviner; fromstok-
hazesthai, to guess at, to aim at; and fromstochostarget, aim, guess). The
modifier stochastic indicates that a subject is viewed as random in some as-
pect. Stochastic is often used in contrast to “deterministic,” which means
that random phenomena are not involved.
More formally, letJbe subset of the non-negative real numbers. Usually
J is the natural numbers 0, 1 , 2 ,...or the non-negative reals{t:t≥ 0 }. J
is the index set of the process, and we usually refer tot∈J as the time
variable. Let Ω be a set, usually called thesample spaceorprobability
space. An elementωof Ω is called asample pointorsample path. Let
S be a set of values, often the real numbers, called thestate space. A
stochastic processis a functionX: (J,Ω)→S, a function of both time
and the sample point to the state space.
Because we are usually interested in the probability of sets of sample
points that lead to a set of outcomes in the state space and not the individual
sample points, the common practice is to suppress the dependence on the
sample point. That is, we usually writeX(t) instead of the more complete
X(t,ω). Furthermore, if the time set is discrete, say the natural numbers,