Reversing : The Hacker's Guide to Reverse Engineering

(ff) #1
Let’s continue this process of elimination in order to determine the func-
tion’s calling convention and observe that the function isn’t taking any regis-
ters from the caller because every register that is accessed is initialized within
the function itself. This shows that this isn’t a _fastcallcalling convention
because _fastcallfunctions receive parameters through ECXand EDX, and
yet these registers are initialized at the very beginning of this function.
The other common calling conventions are stdcalland the C++ member
function calling convention. You know that this is not a C++ member function
because you have its name from the export directory, and you know that it is
undecorated. C++ functions are always decoratedwith the name of their class
and the exact type of each parameter they receive. It is easy to detect decorated
C++ names because they usually include numerous nonalphanumeric charac-
ters and more than one name (class name and method name at the minimum).
By process of elimination you’ve established that the function is an stdcall,
and you now know that the number 14 after the RETinstruction tells you how
many parameters it receives. In this case, OllyDbg outputs hexadecimal num-
bers, so 14 in hexadecimal equals 20 in decimal. Because you’re working in a
32-bit environment parameters are aligned to 32 bits, which are equivalent to
4 bytes, so you can assume that the function receives five parameters. It is possi-
ble that one of these parameters would be larger than 4 bytes, in which case the
function receives less than five parameters, but it can’t possibly be more than
five because parameters are 32-bit aligned.
In looking at the function’s prologue, you can see that it uses a standard EBP
stack frame. The current value of EBPis saved on the stack, and EBPtakes the
value of ESP. This allows for convenient access to the parameters that were
passed on the stack regardless of the current value of ESPwhile running the
function (ESPconstantly changes whenever the function pushes parameters
into the stack while calling other functions). In this very popular layout, the
first parameter is placed at [EBP + 8], the second at [ebp + c], and so on. If
you’re not sure why that is so please refer to Appendix C for a detailed expla-
nation of stack frames.
Typically, a function would also allocate room for local variables by sub-
tracting ESPwith the number of bytes needed for local variable storage, but
this doesn’t happen in this function, indicating that the function doesn’t store
any local variables in the stack.
Let us go over the function from Listing 5.1 instruction by instruction and
see what it does. As I mentioned earlier, you might want to do this using live
analysis by stepping through this code in the debugger and actually seeing
what happens during its execution using GenericTable.EXE. If you’re feel-
ing pretty comfortable with assembly language by now, you could probably
just read through the code in Listing 5.1 without using GenericTable.EXE.
Let’s dig further into the function and determine how it works and what it
does.

148 Chapter 5

Free download pdf