take over the program in question) before the function reaches the cookie verifi-
cation code. There are several different tricks that will work in different cases.
One option is to try and overwrite the area in the stack where parameters were
passed to the function. This trick works for functions that use stack parameters
for returning values to their callers, and is typically implemented by having the
caller pass a memory address as a parameter and by having the callee write back
into that memory address.
The idea is that when a function has a buffer overflow bug, the memory
address used for returning values to the caller (assuming that the function
does that) can be overwritten using a specially crafted buffer, which would get
the function to overwrite a memory address chosen by the attacker (because
the function takes that address and writes to it). By being able to write data to
an arbitrary address in memory attackers can sometimes gain control of the
process before the stack-checking code finds out that a buffer overflow had
occurred. In order to do that, attackers must locate a function that passes val-
ues back to the caller using parameters and that has an overflow bug. Then in
order to exploit such a vulnerability, they must figure out an address to write
to in memory that would allow them to run their own code before the process
is terminated by the stack-checking code. This address is usually some kind of
global address that controls which code is executed when stack checking fails.
As you can see, exploiting programs that have stack-checking mechanisms
embedded into them is not as easy as exploiting simple buffer overflow bugs.
This means that even though it doesn’t completely eliminate the problem,
stack checking does somewhat reduce the total number of possible exploits in
a program.Nonexecutable Memory
This discussion wouldn’t be complete without mentioning one other weapon
that helps fight buffer overflows: nonexecutable memory. Certain processors
provide support for defining memory pages as nonexecutable, which means
that they can only be used for storing data, and that the processor will not run
code stored in them. The operating system can then mark stack and data pages
as nonexecutable, which prevents an attacker from running code on them
using a buffer overflow.
At the time of writing, many new processors already support this function-
ality (including recent versions of Intel and AMD processors, and the IA-64
Intel processors), and so do many operating systems (including Windows XP
Service Pack 2 and above, Solaris 2.6 and above, and several patches imple-
mented for the Linux kernel).
Needless to say, nonexecutable memory doesn’t exactly invalidate the
whole concept of buffer overflow attacks. It is quite possible for attackers to254 Chapter 7