Reverse Engineering for Beginners

CHAPTER 64. ARGUMENTS PASSING METHODS (CALLING CONVENTIONS) CHAPTER 64. ARGUMENTS PASSING METHODS (CALLING CONVENTIONS)

Listing 64.5: Optimizing MSVC 2010 /Ob0

_c$ = 8 ; size = 4
@f3@12 PROC
; _a$ = ecx
; _b$ = edx
mov eax, ecx
imul eax, edx
add eax, DWORD PTR _c$[esp-4]
ret 4
@f3@12 ENDP

; ...

mov edx, 2

push 3

lea ecx, DWORD PTR [edx-1]

call @f3@12

push eax

push OFFSET $SG81390

call _printf

add esp, 8

We see that thecalleereturnsSPby using theRETNinstruction with an operand. Which implies that the number of
arguments can be deduced easily here as well.

64.3.1 GCC regparm

It is the evolution offastcall^2 in some sense. With the-mregparmoption it is possible to set how many arguments are to
be passed via registers (3 is the maximum). Thus, theEAX,EDXandECXregisters are to be used.

Of course, if the number the of arguments is less than 3, not all 3 registers are to be used.

Thecallerrestores thestack pointerto its initial state.

For example, see (19.1.1 on page 290).

64.3.2 Watcom/OpenWatcom.

Here it is called “register calling convention”. The first 4 arguments are passed via theEAX,EDX,EBXandECXregisters. All
the rest—via the stack. These functions has an underscore appended to the function name in order to distinguish them from
those having a different calling convention.

64.4 thiscall.

This is passing the object’sthispointer to the function-method, in C++.

In MSVC,thisis usually passed in theECXregister.

In GCC, thethispointer is passed as the first function-method argument. Thus it will be very visible that internally: all
function-methods have an extra argument.

For an example, see (51.1.1 on page 522).

64.5 x86-64

64.5.1 Windows x64

The method of for passing arguments in Win64 somewhat resemblesfastcall. The first 4 arguments are passed viaRCX,
RDX,R8andR9, the rest—via the stack. Thecalleralso must prepare space for 32 bytes or 4 64-bit values, so then thecallee
can save there the first 4 arguments. Short functions may use the arguments’ values just from the registers, but larger ones
may save their values for further use.

(^2) http://go.yurichev.com/17040