Reversing : The Hacker's Guide to Reverse Engineering

(ff) #1

Control Flow Transformations


Control flow transformations are transformations that alter the order and flow
of a program in a way that reduces its human readability. In “Manufacturing
Cheap, Resilient, and Stealthy Opaque Constructs” by Christian Collberg, Clark
Thomborson, and Douglas Low [Collberg1], control flow transformations are
categorized as computation transformations, aggregation transformations, and order-
ing transformations.
Computation transformations are aimed at reducing the readability of the
code by modifying the program’s original control flow structure in ways that
make for a functionally equivalent program that is far more difficult to trans-
late back into a high-level language. This is can be done either by removing
control flow information from the program or by adding new control flow
statements that complicate the program and cannot be easily translated into a
high-level language.
Aggregation transformations destroy the high-level structure of the pro-
gram by breaking the high-level abstractions created by the programmer while
the program was being written. The basic idea is to break such abstractions so
that the high-level organization of the code becomes senseless.
Ordering transformations are somewhat less powerful transformations that
randomize (as much as possible) the order of operations in a program so that
its readability is reduced.

Opaque Predicates


Opaque predicates are a fundamental building block for control flow transfor-
mations. I’ve already introduced some trivial opaque predicates in the previous
section on antidisassembling techniques. The idea is to create a logical state-
ment whose outcome is constant and is known in advance. Consider, for exam-
ple the statement if (x + 1 == x). This statement will obviously never be
satisfied and can be used to confuse reversers and automated decompilation
tools into thinking that the statement is actually a valid part of the program.
With such a simple statement, it is going to be quite easy for both humans
and machines to figure out that this is a false statement. The objective is to cre-
ate opaque predicates that would be difficult to distinguish from the actual
program code and whose behavior would be difficult to predict without actu-
ally stepping into the code. The interesting thing about opaque predicates (and
about several other aspects of code obfuscation as well) is that confusing an
automated deobfuscator is often an entirely different problem from confusing
a human reverser.
Consider for example the concurrency-based opaque predicates suggested
in [Collberg1]. The idea is to create one or more threads that are responsible for

346 Chapter 10

Free download pdf