Reversing : The Hacker's Guide to Reverse Engineering

(ff) #1
other strings, so that internal cross-references are not severed. The typical
strategy is to have a program go over the executable after it is created and just
rename all internal names to meaningless strings.

Code Encryption


Encryption of program code is a common method for preventing static analy-
sis. It is accomplished by encrypting the program at some point after it is com-
piled (before it is shipped to the customer) and embedding some sort of
decryption code inside the executable. Unfortunately, this approach usually
creates nothing but inconvenience for the skillful reverser because in most
cases everything required for the decryption of the program must reside inside
the executable. This includes the decryption logic, and, more importantly, the
decryption key.
Additionally, the program must decrypt the code in runtime before it is exe-
cuted, which means that a decrypted copy of the program or parts of it must
reside in memory during runtime (otherwise the program just wouldn’t be
able to run).
Still, code encryption is a commonly used technique for hindering static
analysis of programs because it significantly complicates the process of ana-
lyzing the program and can sometimes force reversers to perform a runtime
analysis of the program. Unfortunately, in most cases, encrypted programs can
be programmatically decrypted using special unpacker programs that are
familiar with the specific encryption algorithm implemented in the program
and can automatically find the key and decrypt the program. Unpackers typi-
cally create a new executable that contains the original program minus the
encryption.
The only way to fight the automatic unpacking of executables (other than to
use separate hardware that stores the decryption key or actually performs the
decryption) is to try and hide the key within the program. One effective tactic
is to use a key that is calculated in runtime, inside the program. Such a key-
generation algorithm could easily be designed that would require a remark-
ably sophisticated unpacker. This could be accomplished by maintining
multiple global variables that are continuously accessed and modified by var-
ious parts of the program. These variables can be used as a part of a complex
mathematical formula at each point where a decryption key is required. Using
live analysis, a reverser could very easily obtain each of those keys, but the
idea is to use so many of them that it would take a while to obtain all of them
and entirely decrypt the program. Because of the complex key generation
algorithm, automatic decryption is (almost) out of the question. It would take
a remarkable global data-flow analysis tool to actually determine what the
keys are going to be.

330 Chapter 10

Free download pdf