Reversing : The Hacker's Guide to Reverse Engineering

(ff) #1
Take the following code snippet as an example:

mov eax, DWORD PTR [esp-4]
mov DWORD PTR [eax], 0
mov DWORD PTR [eax+4], 1
mov DWORD PTR [eax+8], 2

The problem with this sequence is that you have no idea whether EAXrep-
resents a pointer to a data structure or an array. Typically, array items are not
accessed using hard-coded indexes, and structure members are, but there are
exceptions. In most cases, the preceding machine code would be produced by
accessing structure members in the following fashion:

void foo1(TESTSTRUCT *pStruct)
{
pStruct->a = FALSE;
pStruct->b = TRUE;
pStruct->c = SOMEFLAG; // SOMEFLAG == 2
}

The problem is that without making too much of an effort I can come up
with at least one other source code sequence that would produce the very
same assembly language code. The obvious case is if EAXrepresents an array
and you access its first three 32-bit items and assign values to them, but that’s
a fairly unusual sequence. As I mentioned earlier, arrays are usually accessed
via loops. This brings us to aggressive loop unrolling performed by some com-
pilers under certain circumstances. In such cases, the compiler might produce
the above assembly language sequence (or one very similar to it) even if the
source code contained a loop. The following source code is an example—when
compiled using the Microsoft C/C++ compiler with the Maximize Speed set-
tings, it produces the assembly language sequence you saw earlier:

void foo2(int *pArray)
{
for (int i = 0; i < 3; i++)
pArray[i] = i;
}

This is another unfortunate (yet somewhat extreme) example of how infor-
mation is lost during the compilation process. From a decompiler’s stand-
point, there is no way of knowing whether EAXrepresents an array or a data
structure. Still, because arrays are rarely accessed using hard-coded offsets,
simply assuming that a pointer calculated using such offsets represents a data
structure would probably work for 99 percent of the code out there.

474 Chapter 13

Free download pdf