Reverse Engineering for Beginners

(avery) #1

CHAPTER 64. ARGUMENTS PASSING METHODS (CALLING CONVENTIONS) CHAPTER 64. ARGUMENTS PASSING METHODS (CALLING CONVENTIONS)


64.5.2 Linux x64


The way arguments are passed in Linux for x86-64 is almost the same as in Windows, but 6 registers are used instead of 4
(RDI,RSI,RDX,RCX,R8,R9) and there is no “scratch space”, although thecalleemay save the register values in the stack,
if it needs/wants to.


Listing 64.8: Optimizing GCC 4.7.3

.LC0:
.string "%d %d %d %d %d %d %d\n"
f1:
sub rsp, 40
mov eax, DWORD PTR [rsp+48]
mov DWORD PTR [rsp+8], r9d
mov r9d, ecx
mov DWORD PTR [rsp], r8d
mov ecx, esi
mov r8d, edx
mov esi, OFFSET FLAT:.LC0
mov edx, edi
mov edi, 1
mov DWORD PTR [rsp+16], eax
xor eax, eax
call __printf_chk
add rsp, 40
ret
main:
sub rsp, 24
mov r9d, 6
mov r8d, 5
mov DWORD PTR [rsp], 7
mov ecx, 4
mov edx, 3
mov esi, 2
mov edi, 1
call f1
add rsp, 24
ret


N.B.: here the values are written into the 32-bit parts of the registers (e.g., EAX) but not in the whole 64-bit register (RAX).
This is because each write to the low 32-bit part of a register automatically clears the high 32 bits. Supposedly, it was
decided in AMD to do so to simplify porting code to x86-64.


64.6 Return values offloatanddoubletype


In all conventions except in Win64, the values of typefloatordoubleare returned via the FPU registerST(0).


In Win64, the values offloatanddoubletypes are returned in the low 32 or 64 bits of theXMM0register.


64.7 Modifying arguments.


Sometimes, C/C++ programmers (not limited to thesePLs, though), may ask, what can happen if they modify the arguments?
The answer is simple: the arguments are stored in the stack, that is where the modification takes place. The calling functions
is not using them after thecallee’s exit (author of these lines have never seen any such case in his practice).


#include <stdio.h>


void f(int a, int b)
{
a=a+b;
printf ("%d\n", a);
};


Listing 64.9: MSVC 2012

_a$ = 8 ; size = 4

Free download pdf