overcome the hurdles imposed by nonexecutable memory systems, as long as
a vulnerable piece of code is found [Designer, Wojtczuk]. The most popular
strategy (often called return-to-libc) is to modify the function’s return address
to point to a well-known function (such as a runtime library function or a sys-
tem API) that helps attackers gain control over the process. This completely
avoids the problem of having a nonexecutable stack, but requires a slightly
more involved exploit.
Heap Overflows
Another type of overflow that can be used for taking control of a program or
of the entire system is the malloc exploitor heap overflow[anonymous], [Kaempf],
[jp]. The general idea is the same as a stack overflow: programs receive data of
an unexpected length and copy it into a buffer that’s too small to contain it.
This causes the program to overwrite whatever it is that follows the heap block
in memory. Typically, heaps are arranged as linked lists, and the pointers to the
next and previous heap blocks are placed either right before or right after the
actual block data. This means that writing past the end of a heap block would
corrupt that linked list in some way. Usually, this causes the program to crash
as soon as the heap manager traverses the linked list (in order to free a block
for example), but when done carefully a heap overflow can be used to take
over a system.
The idea is that attackers can take advantage of the heap’s linked-list structure
in order to overwrite some memory address in the process’s address space.
Implementing such attacks can be quite complicated, but the basic idea is fairly
straightforward. Because each block in the linked list has “next” and “prev”
members, it is possible to overwrite these members in a way that would allow
the attacker to write an arbitrary value into an arbitrary address in memory.
Think of what takes place when an element is removed from a doubly
linked list. The system must correct the links in the two adjacent items on the
list (both the previous item and the next item), so that they correctly link to one
another, and not to the item you’re currently deleting. This means that when
the item is removed, the code will write the address of the next member into
the previous item’s header (it will take both addresses from the header of item
currently being deleted), and the address of the prev item into the next item’s
header (again, the addresses will be taken from the item currently being
deleted). It’s not easy, but by carefully overwriting the values of these next and
prev members in one item on the list, attackers can in some cases manage to
overwrite strategic memory addresses in the process address space. Of course,
the overwrite doesn’t take place immediately—it only happens when the over-
written item is freed.
Auditing Program Binaries 255