Data Compression
Richard BriceData reduction or compression techniques are important because universal laws put a
premium on information. You couldn’t read all the books in the world, neither could
you store them. You might make a start on reading every book by making it a team
effort. In other words, you might tackle the problem with more than one brain and one
pair of eyes. In communication theory terms, this approach is known as increasing
the channel capacity by broadening the bandwidth. But you wouldn’t have an infi nite
number of people at your disposal unless you had an infi nite amount of money to pay
them! Likewise no one has an infi nite channel capacity or an infi nite bandwidth at their
disposal. The similar argument applies to storage. Stated axiomatically: information, in
all its forms, is using up valuable resources, so the more effi ciently we can send it and
store it the better. That’s where compression comes in.
If I say to you, “ Wow, I had a bad night, the baby cried from three ’ til six! ” You
understand perfectly what I mean because you know what a baby crying sounds like.
I might alternatively have said, “ Wow, I had a bad night, the baby did this; wah, bwah,
bwah, wah ... ” and continue doing it for 3 h. Try it. You’ll fi nd you lose a lot of friends
because nobody needs to have it demonstrated. Most of the 3-h impersonation is
superfl uous. The second message is said to have a high level of redundancy in the terms
of communication theory. The trick performed by any compression system is sorting out
the necessary information content—sometimes called the entropy—from the redundancy.
(If, like me, you fi nd it diffi cult to comprehend the use of entropy in this context consider
this: entropy refers here to a lack of pattern; to disorder. Everything in a signal that has a
pattern is, by defi nition, predictable and therefore redundant. Only those parts of the signal
that possess no pattern are unpredictable and therefore represent necessary information.)
CHAPTER 19