Vulnerabilities
A vulnerability is essentially a bug or flaw in a program that compromises the
security of the program and usually of the entire computer on which it is run-
ning. Basically, a vulnerability is a flaw in the program that might allow mali-
cious intruders to take advantage of it. In most cases, vulnerabilities start with
code that takes information from the outside world. This can be any type of
user input such as the command-line parameters that programs receive, a file
loaded into the program, or a packet of data sent over the network.
The basic idea is simple—feed the program unexpected input (meaning
input that the programmer didn’t think it was ever going to be fed) and get it
to stray from its normal execution path. A crude way to exploit a vulnerability
is to simply get the program to crash. This is typically the easiest objective
because in many cases simply feeding the program exceptionally large ran-
dom blocks of data does the trick.
But crashing a program is just the beginning. The art of finding and exploit-
ing vulnerabilities gets truly interesting when attackers aim to take control of
the program and get it to run their own code. This requires an entirely differ-
ent level of sophistication, because in order to take control of a program attack-
ers must feed it very specific data.
In many cases, vulnerabilities put entire networks at risk because penetrat-
ing the outer shell of a network frequently means that you’ve crossed the last
line of defense.
The following sections describe the most common vulnerabilities found in
the average program and demonstrate how such vulnerabilities can be utilized
by attackers. You’ll also find examples of how these vulnerabilities can be
found when analyzing assembly language code.
Stack Overflows
Stack overflows (also known as stack-smashing attacks after the well-known
Phrack paper, [Aleph1]) have been around for years and are by far the most
popular type of program vulnerability. Basically, stack overflow exploits take
advantage of the fact that programs (and particularly those written in C-based
languages) frequently neglect to perform bounds checking on incoming data.
A simple stack overflow vulnerability can be created when a program
receives data from the outside world, either as user input directly or through a
network connection, and naively copies that data onto the stack without
checking its length. The problem is that stack variables always have a fixed
size, because the offsets generated by the compiler for accessing those vari-
ables are predetermined and hard-coded into the machine code. This means
that a program can’t dynamically allocate stack space based on the amount of
Auditing Program Binaries 245