1.22. MANIPULATING SPECIFIC BIT(S)
int handle;handle=open ("file", O_RDWR | O_CREAT);
};
We get:
Listing 1.265: GCC 4.4.1
public main
main proc near
var_20 = dword ptr -20h
var_1C = dword ptr -1Ch
var_4 = dword ptr -4
push ebp
mov ebp, esp
and esp, 0FFFFFFF0h
sub esp, 20h
mov [esp+20h+var_1C], 42h
mov [esp+20h+var_20], offset aFile ; "file"
call _open
mov [esp+20h+var_4], eax
leave
retn
main endp
If we take a look in theopen()function in thelibc.so.6library, it is only a syscall:
Listing 1.266: open() (libc.so.6).text:000BE69B mov edx, [esp+4+mode] ; mode
.text:000BE69F mov ecx, [esp+4+flags] ; flags
.text:000BE6A3 mov ebx, [esp+4+filename] ; filename
.text:000BE6A7 mov eax, 5
.text:000BE6AC int 80h ; LINUX - sys_open
So, the bit fields foropen()are apparently checked somewhere in the Linux kernel.
Of course, it is easy to download both Glibc and the Linux kernel source code, but we are interested in
understanding the matter without it.
So, as of Linux 2.6, when thesys_opensyscall is called, control eventually passes todo_sys_open, and
from there—to thedo_filp_open()function (it’s located in the kernel source tree infs/namei.c).
N.B. Aside from passing arguments via the stack, there is also a method of passing some of them via
registers. This is also called fastcall (6.1.3 on page 735). This works faster since CPU does not need to
access the stack in memory to read argument values. GCC has the optionregparm^146 , through which it’s
possible to set the number of arguments that can be passed via registers.
The Linux 2.6 kernel is compiled with-mregparm=3option^147148.
What this means to us is that the first 3 arguments are to be passed via registersEAX,EDXandECX, and
the rest via the stack. Of course, if the number of arguments is less than 3, only part of registers set is to
be used.
So, let’s download Linux Kernel 2.6.31, compile it in Ubuntu:make vmlinux, open it inIDA, and find the
do_filp_open()function. At the beginning, we see (the comments are mine):
Listing 1.267: do_filp_open() (linux kernel 2.6.31)do_filp_open proc near
...
push ebp
mov ebp, esp
push edi
push esi
push ebx
(^146) ohse.de/uwe/articles/gcc-attributes.html#func-regparm
(^147) kernelnewbies.org/Linux_2_6_20#head-042c62f290834eb1fe0a1942bbf5bb9a4accbc8f
(^148) See alsoarch/x86/include/asm/calling.hfile in kernel tree