199Most of this book describes how to reverse engineer programs in order to get
an insight into their internal workings. This chapter discusses a slightly differ-
ent aspect of this craft: the general process of deciphering program data. This
data can be an undocumented file format, a network protocol, and so on. The
process of deciphering such data to the point where it is possible to actually use
it for the creation of programs that can accept and produce compatible data is
another branch of reverse engineering that is often referred to as data reverse
engineering. This chapter demonstrates data reverse-engineering techniques
and shows what can be done with them.
The most common reason for performing any kind of data reverse engineer-
ing is to achieve interoperability with a third party’s software product. There are
countless commercial products out there that use proprietary, undocumented
data formats. These can be undocumented file formats or networking protocols
that cannot be accessed by any program other than those written by the original
owner of the format—no one else knows the details of the proprietary format.
This is a major inconvenience to end users because they cannot easily share their
files with people that use a competing program—only the products developed
by the owner of the file format can access the proprietary file format.
This is where data reverse engineering comes into play. Using data reverse
engineering techniques it is possible to obtain that missing information
regarding a proprietary data format, and write code that reads or even gener-
ates data in the proprietary format. There are numerous real-world examples