Reverse Engineering for Beginners

(avery) #1
CHAPTER 64. ARGUMENTS PASSING METHODS (CALLING CONVENTIONS) CHAPTER 64. ARGUMENTS PASSING METHODS (CALLING CONVENTIONS)

Chapter 64


Arguments passing methods (calling conventions)


64.1 cdecl


This is the most popular method for passing arguments to functions in the C/C++ languages.

The glscaller also must return the value of thestack pointer(ESP) to its initial state after thecalleefunction exits.

Listing 64.1: cdecl
push arg3
push arg2
push arg1
call function
add esp, 12 ; returns ESP

64.2 stdcall


It’s almost the same ascdecl, with the exception that thecalleemust setESPto the initial state by executing theRET x
instruction instead ofRET, wherex = arguments number * sizeof(int)^1. Thecalleris not adjusting thestack
pointer, there are noadd esp, xinstruction.

Listing 64.2: stdcall
push arg3
push arg2
push arg1
call function

function:
... do something ...
ret 12

The method is ubiquitous in win32 standard libraries, but not in win64 (see below about win64).

For example, we can take the function from8.1 on page 88and change it slightly by adding the__stdcallmodifier:

int __stdcall f2 (int a, int b, int c)
{
return a*b+c;
};

It is to be compiled in almost the same way as8.2 on page 88, but you will seeRET 12instead ofRET.SPis not update in
thecaller.

As a consequence, the number of function arguments can be easily deduced from theRETN ninstruction: just dividenby
4.

Listing 64.3: MSVC 2010
_a$ = 8 ; size = 4

(^1) The size of aninttype variable is 4 in x86 systems and 8 in x64 systems

Free download pdf