Reverse Engineering for Beginners

APPENDIX A. X86 APPENDIX A. X86

;

rcm40: sub esi,4 ; back up

sub edi,4 ; back up

mov ecx,5 ; ensure that ecx doesn't count out

repe cmpsb ; find mismatch byte

;

; When we come to rcm50, esi points to the byte after the one that

; did not match, which is TWO after the last byte that did match.

;

rcm50: dec esi ; back up

sub esi,RcmSource1 ; compute bytes that matched

mov eax,esi ;

pop edi ; restore registers

pop esi ;

stdRET _RtlCompareMemory

stdENDP _RtlCompareMemory

N.B.: this function uses a 32-bit word comparison (CMPSD) if the block size is a multiple of 4, or per-byte comparison

(CMPSB) otherwise.

CPUIDget information about theCPU’s features. see also: (21.6.1 on page 351).

DIVunsigned division

IDIVsigned division

INT(M):INT xis analogous toPUSHF; CALL dword ptr [x*4]in 16-bit environment. It was widely used in MS-
DOS, functioning as a syscall vector. The registers AX/BX/CX/DX/SI/DI were filled with the arguments and then the
flow jumped to the address in the Interrupt Vector Table (located at the beginning of the address space). It was
popular because INT has a short opcode (2 bytes) and the program which needs some MS-DOS services is not bother
to determine the address of the service’s entry point. The interrupt handler returns the control flow to caller using the
IRET instruction.

The most busy MS-DOS interrupt number was 0x21, serving a huge part of itsAPI. See also: [Bro] for the most

comprehensive interrupt lists and other MS-DOS information.

In the post-MS-DOS era, this instruction was still used as syscall both in Linux and Windows (66 on page 661), but

was later replaced by the SYSENTER or SYSCALL instructions.

INT 3(M): this instruction is somewhat close toINT, it has its own 1-byte opcode (0xCC), and is actively used while debug-
ging. Often, the debuggers just write the0xCCbyte at the address of the breakpoint to be set, and when an exception
is raised, the original byte is restored and the original instruction at this address is re-executed.
As ofWindows NT, anEXCEPTION_BREAKPOINTexception is to be raised when theCPUexecutes this instruction.
This debugging event may be intercepted and handled by a host debugger, if one is loaded. If it is not loaded, Windows
offers to run one of the registered system debuggers. IfMSVS^5 is installed, its debugger may be loaded and connected
to the process. In order to protect fromreverse engineering, a lot of anti-debugging methods check integrity of the
loaded code.

MSVChascompiler intrinsicfor the instruction:__debugbreak()^6.

There is also a win32 function in kernel32.dll namedDebugBreak()^7 , which also executesINT 3.

IN(M) input data from port. The instruction usually can be seen in OS drivers or in old MS-DOS code, for example (78.3
on page 746).

IRET: was used in the MS-DOS environment for returning from an interrupt handler after it was called by the INT instruction.
Equivalent toPOP tmp; POPF; JMP tmp.

LOOP(M)decrementCX/ECX/RCX, jump if it is still not zero.

OUT(M) output data to port. The instruction usually can be seen in OS drivers or in old MS-DOS code, for example (78.3
on page 746).

POPA(M) restores values of (R|E)DI, (R|E)SI, (R|E)BP, (R|E)BX, (R|E)DX, (R|E)CX, (R|E)AX registers from the stack.

POPCNTpopulation count. Counts the number of 1 bits in the value.AKA“hamming weight”.AKA“NSA instruction” due to
some rumors:

(^5) Microsoft Visual Studio
(^6) MSDN
(^7) MSDN