Reversing : The Hacker's Guide to Reverse Engineering

Let’s step into this function at 401D18to determine how it produces the

decryption key. As soon as you enter this function, you realize that you have a

bit of a problem: It is also encrypted. Of course, the question now is where

does the decryption key for this function come from? There are two code

sequences that appear to be relevant. When the function starts, it performs the

following:

00401D1F MOV EAX,DWORD PTR SS:[EBP+8]

00401D22 IMUL EAX,DWORD PTR DS:[406020]

00401D29 MOV DWORD PTR SS:[EBP-10],EAX

This sequence takes the low-order word of the name integer that was pro-

duced earlier and multiplies it with a global variable at [406020]. If you go

back to the function that obtained the volume serial number, you will see that

it was stored at [406020]. So, Defender is multiplying the low part of the

name integer with the volume serial number, and storing the result in [EBP-

10]. The next sequence that appears related is part of the decryption loop:

00401D7B MOV EAX,DWORD PTR SS:[EBP+10]

00401D7E MOV ECX,DWORD PTR SS:[EBP-10]

00401D81 SUB ECX,EAX

00401D83 MOV EAX,DWORD PTR SS:[EBP-28]

00401D86 XOR ECX,DWORD PTR DS:[EAX]

This sequence subtracts the parameter at [EBP+10]from the result of the

previous multiplication, and XORs that value against the encrypted function!

Essentially Defender is doing Key = (NameInt * VolumeSerial) – LOWPART(Seri-

alNumber). Smells like trouble! Let the decryption routine complete the decryp-

tion, and try to step into the decrypted code. Here’s what the beginning of the

decrypted code looks like (this is quite random—your milage may vary).

00401E32 PUSHFD

00401E33 AAS

00401E34 ADD BYTE PTR DS:[EDI],-22

00401E37 AND DH,BYTE PTR DS:[EAX+B84CCD0]

00401E3D LODS BYTE PTR DS:[ESI]

00401E3E INS DWORD PTR ES:[EDI],DX

It is quite easy to see that this is meaningless junk. It looks like the decryp-

tion failed. But still, it looks like Defender is going to try to execute this code!

What happens now really depends on which debugger you’re dealing with,

but Defender doesn’t just go away. Instead it prints its lovely “Sorry... Bad

Key.” message. It looks like the top-level exception handler installed earlier is

the one generating this message. Defender is just crashing because of the bad

code in the function you just studied, and the exception handler is printing the

message.

408 Chapter 11