Assembly Language for Beginners

1.20. ARRAYS

(gdb)

The register values are slightly different than in win32 example, since the stack layout is slightly different

too.

1.20.3 Buffer overflow protection methods.

There are several methods to protect against this scourge, regardless of the C/C++ programmers’ negli-

gence. MSVC has options like^135 :

/RTCs Stack Frame runtime checking

/GZ Enable stack checks (/RTCs)

One of the methods is to write a random value between the local variables in stack at function prologue

and to check it in function epilogue before the function exits. If value is not the same, do not execute the

last instructionRET, but stop (or hang). The process will halt, but that is much better than a remote attack

to your host.

This random value is called a “canary” sometimes, it is related to the miners’ canary^136 , they were used

by miners in the past days in order to detect poisonous gases quickly.

Canaries are very sensitive to mine gases, they become very agitated in case of danger, or even die.

If we compile our very simple array example (1.20.1 on page 268) inMSVCwith RTC1 and RTCs option,
you can see a call to@_RTC_CheckStackVars@8a function at the end of the function that checks if the
“canary” is correct.

Let’s see how GCC handles this. Let’s take analloca()(1.7.2 on page 35) example:

#ifdef __GNUC__

#include <alloca.h> // GCC

#else

#include <malloc.h> // MSVC

#endif

#include <stdio.h>

void f()

{

char *buf=(char*)alloca (600);

#ifdef __GNUC__

snprintf (buf, 600, "hi! %d, %d, %d\n", 1, 2, 3); // GCC

#else

_snprintf (buf, 600, "hi! %d, %d, %d\n", 1, 2, 3); // MSVC

#endif

puts (buf);

};

By default, without any additional options, GCC 4.7.3 inserts a “canary” check into the code:

Listing 1.231: GCC 4.7.3

.LC0:

.string "hi! %d, %d, %d\n"

f:

push ebp

mov ebp, esp

push ebx

sub esp, 676

lea ebx, [esp+39]

and ebx, -16

mov DWORD PTR [esp+20], 3

mov DWORD PTR [esp+16], 2

mov DWORD PTR [esp+12], 1

mov DWORD PTR [esp+8], OFFSET FLAT:.LC0 ; "hi! %d, %d, %d\n"

mov DWORD PTR [esp+4], 600

(^135) compiler-side buffer overflow protection methods:wikipedia.org/wiki/Buffer_overflow_protection
(^136) wikipedia.org/wiki/Domestic_canary#Miner.27s_canary