13 Sequential Data
So far in this book, we have focussed primarily on sets of data points that were as-
sumed to be independent and identically distributed (i.i.d.). This assumption allowed
us to express the likelihood function as the product over all data points of the prob-
ability distribution evaluated at each data point. For many applications, however,
the i.i.d. assumption will be a poor one. Here we consider a particularly important
class of such data sets, namely those that describe sequential data. These often arise
through measurement of time series, for example the rainfall measurements on suc-
cessive days at a particular location, or the daily values of a currency exchange rate,
or the acoustic features at successive time frames used for speech recognition. An
example involving speech data is shown in Figure 13.1. Sequential data can also
arise in contexts other than time series, for example the sequence of nucleotide base
pairs along a strand of DNA or the sequence of characters in an English sentence.
For convenience, we shall sometimes refer to ‘past’ and ‘future’ observations in a
sequence. However, the models explored in this chapter are equally applicable to all