Reversing : The Hacker's Guide to Reverse Engineering

Beyond the mere additional complexity introduced by adding additional
logic and arithmetic to a program, an obfuscating transformation must be
resilient (meaning that it cannot be easily undone). Because many of these trans-
formations add irrelevant instructions that don’t really produce valuable data,
it is possible to create deobfuscators. A deobfuscator is a program that imple-
ments various data-flow analysis algorithms on an obfuscated program which
sometimes enable it to separate the wheat from the chaff and automatically
remove all irrelevant instructions and restore the code’s original structure. Cre-
ating resilient obfuscation transformations that are resistant to deobfuscation is
a major challenge and is the primary goal of many obfuscators.
Finally, an obfuscating transformation will typically have an associated cost.
This can be in the form of larger code, slower execution times, or increased
memory runtime consumption. It is important to realize that some transfor-
mations do not incur any kind of runtime costs, because they involve a simple
reorganization of the program that is transparent to the machine, but makes
the program less human-readable.
In the following sections, I will be going over the common obfuscating
transformations. Most of these transformations were meant to be applied pro-
grammatically by running an obfuscator on an existing program, either at the
source code or the binary level. Still, many of these transformations can be
applied manually, while the program is being written or afterward, before it is
shipped to end users. Automatic obfuscation is obviously far more effective
because it can obfuscate the entire program and not just small parts of it. Addi-
tionally, automatic obfuscation is typically performed after the program is
compiled, which means that the original source code is not made any less
readable (as is the case when obfuscation is performed manually).

Antireversing Techniques 345

OBFUSCATION TOOLS

Let’s take a quick look at the existing obfuscation tools that can be used to

obfuscate programs on the fly. There are quite a few bytecode obfuscators for

Java and .NET, and I will be discussing and evaluating some of them in Chapter

12. As for obfuscation of native IA-32 code, there aren’t that many generic tools
    that process entire executables and effectively obfuscate them. One notable
    product that is quite powerful is EXECryptor by StrongBit Technology
    (www.strongbit.com). EXECryptor processes PE executables and applies a
    variety of obfuscating transformations on the machine code. Code obfuscated
    by EXECryptor really becomes significantly more difficult to reverse compared
    to plain IA-32 code. Another powerful technology is the StarForce suite of copy
    protection products, developed by StarForce Technologies (www.star-force.
    com). The StarForce products are more than just powerful obfuscation products:
    they are full-blown copy protection products that provide either hardware-
    based or pure software-based copy protection functionality.