Reversing : The Hacker's Guide to Reverse Engineering

In looking at Table A.2, the ground rules for identifying the results of

unsigned integer comparisons become clear, and it’s obvious that unsigned

operands are easier to deal with. Here’s a quick summary of the basic rules:

■■ Anytime ZF is set you know that the subtraction resulted in a zero,

which means that the operands are equal.

■■ When both flags are zero, you know that the first operand is greater

than the second, because you have a positive result and no overflow.

■■ When you have an overflow you know that the second operand is

greater than the first, because the result must be too low in order to be

represented by the destination operand.

The Conditional Codes

Conditional codes are suffixes added to certain conditional instructions in

order to define the conditions governing their execution.

It is important for reversers to understand these mnemonics because virtu-

ally every conditional code sequence will include one or more of them. Some-

times their meaning will be very intuitive—take a look at the following code:

cmp

eax, 7

je

SomePlace

In this example, it is obvious that JE(which is jump if equal) will cause a

jump to SomePlaceif EAXequals 7. This is one of the more obvious cases

where understanding the specifics of instructions such as CMPand of the con-

ditional codes is really unnecessary. Unfortunately for us reversers, there are

quite a few cases where the conditional codes are used in unintuitive ways.

Understanding how the conditional codes use the flags is important for prop-

erly understanding program logic. The following sections list each condition

code and explain which flags it uses and why.

The conditional codes listed in the following sections are listed as standalone

codes, even though they are normally used as instruction suffixes to

conditional instructions. Conditional codes are never used alone.

Signed Conditional Codes

Table A.3 presents the IA-32 conditional codes defined for signed operands.

Note that in all signed conditional codes overflows are detected using the

Deciphering Code Structures 483

21_574817 appa.qxd 3/16/05 8:52 PM Page 483