Reversing : The Hacker's Guide to Reverse Engineering

With this view, you can immediately see a somewhat improved picture. The

first three DWORDs are obviously some kind of 32-bit fields. The last four

DWORDs are not as obvious, and seem to be some kind of a random 16-byte

sequence. This is easy to tell because they do not contain text (you would have

seen that in the previous dump), and they are not pointers or offsets into the

file (the numbers are far too large, and some of them are not 32-bit aligned).

This is a classic case where stepping into the code that deciphers this data

should really simplify the process of deciphering the file format.

The code that actually reads the file table and displays the file list is shown in

Listing 6.6 and is actually quite simple to analyze because the fields that it reads

are both printed into the screen, so it’s very easy to tell what they stand for. Let’s

go back to that code sequence and see what it’s doing with this file entry.

00401A60 MOV ESI,SS:[ESP+10]

00401A64 ADD ESI,8

00401A67 MOV DWORD PTR SS:[ESP+14],1A

00401A6F NOP

00401A70 MOV EAX,DS:[ESI]

00401A72 TEST EAX,EAX

00401A74 JE SHORT cryptex.00401A9A

00401A76 MOV EDX,EAX

00401A78 SHL EDX,0A

00401A7B SUB EDX,EAX

00401A7D ADD EDX,EDX

00401A7F LEA ECX,DS:[ESI+14]

00401A82 ADD EDX,EDX

00401A84 PUSH ECX

00401A85 SHR EDX,0A

This sequence starts out by loading ESIwith the newly decrypted block’s

starting address, adding 8 to that, and reading a 32-bit member at that address

into EAX. If you go back to the previous memory dump, you’ll see that the

third DWORD contains 00000001. At this point, the code confirms that EAX

is nonzero, and proceeds to perform an interesting series of arithmetic opera-

tions on it.

First, EDXis shifted left by 0xA(10) bits, then the original value (from EAX)

is subtracted from the result. At that point, the value of EDXis added to itself

(which is the equivalent of multiplying it by two). This operation is performed

again in 00401A82, and is followed by a right-shift of 0xA(10) bits. Now let’s

go over these operations step by step and try to determine their purpose.

1. EDXis shifted left by 10, which is equivalent to edx = edx ×1,024.


2. The original number at EAXis subtracted from EDX. This means that
   instead of 1,024, you have essentially performed edx = edx ×1,024 – edx,
   which is the equivalent of edx = edx ×1,023.

224 Chapter 6