Reverse Engineering for Beginners

(avery) #1

CHAPTER 18. ARRAYS CHAPTER 18. ARRAYS


18.3 Buffer overflow protection methods.


There are several methods to protect against this scourge, regardless of the C/C++ programmers’ negligence. MSVC has
options like^6 :


/RTCs Stack Frame runtime checking
/GZ Enable stack checks (/RTCs)

One of the methods is to write a random value between the local variables in stack at function prologue and to check it in
function epilogue before the function exits. If value is not the same, do not execute the last instructionRET, but stop (or
hang). The process will halt, but that is much better than a remote attack to your host.


This random value is called a “canary” sometimes, it is related to the miners’ canary^7 , they were used by miners in the past
days in order to detect poisonous gases quickly. Canaries are very sensitive to mine gases, they become very agitated in
case of danger, or even die.


If we compile our very simple array example (18.1 on page 253) inMSVCwith RTC1 and RTCs option, you can see a call to
@_RTC_CheckStackVars@8a function at the end of the function that checks if the “canary” is correct.


Let’s see how GCC handles this. Let’s take analloca()(5.2.4 on page 26) example:


#ifdef GNUC
#include <alloca.h> // GCC
#else
#include <malloc.h> // MSVC
#endif
#include <stdio.h>


void f()
{
char buf=(char)alloca (600);
#ifdef GNUC
snprintf (buf, 600, "hi! %d, %d, %d\n", 1, 2, 3); // GCC
#else
_snprintf (buf, 600, "hi! %d, %d, %d\n", 1, 2, 3); // MSVC
#endif


puts (buf);
};


By default, without any additional options, GCC 4.7.3 inserts a “canary” check into the code:


Listing 18.7: GCC 4.7.3

.LC0:
.string "hi! %d, %d, %d\n"
f:
push ebp
mov ebp, esp
push ebx
sub esp, 676
lea ebx, [esp+39]
and ebx, -16
mov DWORD PTR [esp+20], 3
mov DWORD PTR [esp+16], 2
mov DWORD PTR [esp+12], 1
mov DWORD PTR [esp+8], OFFSET FLAT:.LC0 ; "hi! %d, %d, %d\n"
mov DWORD PTR [esp+4], 600
mov DWORD PTR [esp], ebx
mov eax, DWORD PTR gs:20 ; canary
mov DWORD PTR [ebp-12], eax
xor eax, eax
call _snprintf
mov DWORD PTR [esp], ebx
call puts
mov eax, DWORD PTR [ebp-12]
xor eax, DWORD PTR gs:20 ; check canary
jne .L5


(^6) compiler-side buffer overflow protection methods:wikipedia.org/wiki/Buffer_overflow_protection
(^7) wikipedia.org/wiki/Domestic_canary#Miner.27s_canary

Free download pdf