Reversing : The Hacker's Guide to Reverse Engineering

call SomeFunc

sub eax, 4

neg eax

sbb eax, eax

and al, -52

add eax, 54

ret

You’ll notice that this sequence also uses the NEG/SBBcombination, except

that this one has somewhat more complex functionality. The sequence starts

by calling a function and subtracting 4 from its return value. It then invokes

NEGand SBBin order to perform a zero test on the result, just as you saw in the

previous example. If after the subtraction the return value from SomeFuncis

zero, SBBwill set EAXto zero. If the subtracted return value is nonzero, SBB

will set EAXto –1 (or 0xffffffffin hexadecimal).

The next two instructions are the clever part of this sequence. Let’s start by

looking at that ANDinstruction. Because SBBis going to set EAXeither to zero

or to 0xffffffff, we can consider the following ANDinstruction to be simi-

lar to a conditional assignment instruction (much like the CMOVinstruction

discussed later). By ANDing EAXwith a constant, the code is essentially saying:

“if the result from SBBis zero, do nothing. If the result is –1, set EAXto the

specified constant.” After doing this, the code unconditionally adds 54 to EAX

and returns to the caller.

The challenge at this point is to try and figure out what this all means. This

sequence is obviously performing some kind of transformation on the return

value of SomeFuncand returning that transformed value to the caller. Let’s try

and analyze the bottom line of this sequence. It looks like the return value is

going to be one of two values: If the outcome of SBBis zero (which means that

SomeFunc’s return value was 4), EAXwill be set to 54. If SBBproduces

0xffffffff, EAXwill be set to 2, because the ANDinstruction will set it to –52,

and the ADDinstruction will bring the value up to 2.

This is a sequence that compares a pair of integers, and produces (without

the use of any branches) one value if the two integers are equal and another

value if they are unequal. The following is a C version of the assembly lan-

guage snippet from earlier:

if (SomeFunc() == 4)

return 54;

else

return 2;

512 Appendix A

21_574817 appa.qxd 3/16/05 8:54 PM Page 512