can be defined as having control of the flow of informationon a system. This con-
trol means that your files stay insideyour computer and out of the hands of
nosy intruders, while malicious code stays outsideof your computer. Needless
to say, there are many other aspects to computer security such as the encryp-
tion of information that does flow in and out of the computer and the different
levels of access rights granted to different users, but these are not as relevant to
our current discussion.
So how does reversing relate to maintaining control of the flow of information
on a system? The idea is that whenever you install any kind of software product,
you are essentially entrusting your computer and all of the data on it to that pro-
gram. There are two levels in which this is true. First of all, by installing a soft-
ware product you are trusting that it is benign and that it doesn’t contain any
malicious components that would intentionally steal or corrupt your data.
Believe it or not, that’s the simpler part of this story.
The place where things truly get fuzzy is when we start talking about how
programs put your system in jeopardy without ever intending to. A simple
bug in any kindof software product could theoretically expose your system to
malicious code that could steal or corrupt your data. Take an image file such as
a JPEG as an example. There are certain types of bugs that could, in some
cases, allow a person to take over your system using a specially crafted image
file. All it would take is a tiny, otherwise harmless bug in your image viewing
program, and that program might inadvertently allow code embedded into
the image file to run. What could that code do? Well, just about anything. It
would most likely download some sort of backdoor program onto your sys-
tem, and pave the way for a full-blown hostile takeover (backdoors and other
types of malicious programs are discussed in Chapter 8).
The purpose of this chapter is to try and define what makes secure code, and
to then demonstrate how we can scan binary executables for these types of
security bugs. Unfortunately, attempting to define what makes secure code
can sometimes be a futile attempt. This fact should be painfully clear to soft-
ware developers who constantly release patches that address vulnerabilities
found in their program. It can be a never-ending journey—a game of cat and
mouse between hackers looking for vulnerabilities and programmers trying to
fix them. Few programs start out as being “totally secure,” and in fact, few pro-
grams ever reach that state.
In this chapter, I will make an attempt to cover the most typical bugs that
turn an otherwise-harmless program into a security risk, and will describe
how such bugs can be located while a program is being reversed. This is by no
means intended to be a complete guide to every possible security hole you
could find in software (and I doubt such guide could ever be written), but sim-
ply to give an idea of the types of problems typically encountered.244 Chapter 7