Reversing : The Hacker's Guide to Reverse Engineering

reversing, and explain their exact meanings. I begin by going over the process

of comparing two operands in assembly language, which is a significant build-

ing block used in almost every logical statement. I then proceed to discuss the

conditional codes in IA-32 assembly language, which are employed in every

conditional instruction in the instruction set.

Comparing Operands

The vast majority of logical statements involve the comparison of two or more

operands. This is usually followed by code that can act differently based on the

result of the comparison. The following sections demonstrate the operand

comparison mechanism in IA-32 assembly language. This process is some-

what different for signed and unsigned operands.

The fundamental instruction for comparing operands is the CMPinstruction.

CMPessentially subtracts the second operand from the first and discards the

result. The processor’s flags are used for notifying the instructions that follow

on the result of the subtraction. As with many other instructions, flags are read

differently depending on whether the operands are signed or unsigned.

If you’re not familiar with the subtleties of IA-32 flags, it is highly

recommended that you go over the “Arithmetic Flags” section in Appendix B

before reading further.

Signed Comparisons

Table A.1 demonstrates the behavior of the CMPinstruction when comparing

signed operands. Remember that the following table also applies to the SUB

instruction.

Table A.1 Signed Subtraction Outcome Table for CMPand SUBInstructions (Xrepresents

the left operand, while Yrepresents the right operand)

RELATION

LEFT RIGHT BETWEEN FLAGS

OPERAND OPERAND OPERANDS AFFECTED COMMENTS

X>= 0 Y>= 0 X= Y OF = 0 SF = 0 ZF = 1 The two

operands are

equal, so the

result is zero.

X> 0 Y>= 0 X> Y OF = 0 SF = 0 ZF = 0 Flags are all zero,

indicating a

positive result,

with no overflow.

480 Appendix A

21_574817 appa.qxd 3/16/05 8:52 PM Page 480