58 CHAPTER 1. BACKGROUND IDEAS
then we usually write the index variable or time variable as a subscript.
ThusXnwould be the usual notation for a stochastic process indexed by the
natural numbers andX(t) is a stochastic process indexed by the non-negative
reals. Because of the randomness, we can think of a stochastic process as
a random sequence if the index set is the natural numbers and a random
function if the time variable is the non-negative reals.
Examples
The most fundamental example of a stochastic process is a coin flip sequence.
The index set is the set of counting numbers, counting the number of the flip.
The sample space is the set of all possible infinite coin flip sequences Ω =
{HHTHTTTHT ...,THTHTTHHT ...,...}. We take the state space to
be the set 1,0 so thatXn= 1 if flipncomes up heads, andXn= 0 if the flip
comes up tails. Then the coin flip stochastic process can be viewed as the set
of all “random” sequences of 1’s and 0’s. An associated random process is to
takeSn=
∑n
j=1Xj. Now the state space is the set of natural numbers. The
stochastic processSncounts the number of heads encountered in the flipping
sequence up to flip numbern.
Alternatively, we can take the same index set, the same probability space
of coin flip sequences and defineYn= 1 if flipncomes up heads, andYn=− 1
if the flip comes up tails. This is just another way to encode the coin flips
now as random sequences of 1’s and−1’s. A more interesting associated
random process is to takeTn=
∑n
j=1Yj. Now the state space is the set of
integers. The stochastic processTngives the position in the integers after
taking a step to the right for a head, and a step to the left for a tail. This
particular stochastic process is usually called asimple random walk. We
can generalize random walk by allowing the state space to be the set of points
with integer coordinates in two-, three- or higher-dimensional space, called
the integer lattice.
Markov Chains
AMarkov chainis sequence of random variablesXj where the indexj
runs through 0, 1 , 2 ,.... The sample space is not specified explicitly, but it
involves a sequence of random selections detailed by the effect in the state
space. The state space can be either a finite or infinite set of discrete states.