What Are We Looking For?
Typically, the search for undocumented code starts with a requirement. What
functionality is missing? Which software component can be expected to offer
this functionality? This is where a general knowledge of the program in ques-
tion comes into play. You need to be aware of the key executable modules that
make up the program and to be familiar with the interfaces between those
modules. Interfaces between binary modules are easy to observe simply by
dumping the import and export directories of those modules (this is described
in detail in Chapter 3).
In this particular case, I have decided to look for an interesting Windows API
to dissect. Knowing that the majority of undocumented user-mode services in
Windows are implemented in NTDLL.DLL(because that’s where the native API
is implemented), I simply dumped the export directory of NTDLL.DLLand
visually scanned that list for groups of APIs that appear related (based on their
names).
Of course, this is a somewhat unusual case. In most cases, you won’t just be
looking for undocumented APIs just because they’re undocumented (unless you
just find it really cool to use undocumented APIs and feel like trying it out) —
you will have a specific feature in mind. In this case, you might want to search
that export directory for relevant keywords. Suppose, for example, that you
want to look for some kind of special memory allocation API. In such a case, you
should just search the export list of NTDLL.DLL(or any DLL in which you sus-
pect your API might be implemented) for some relevant keywords such as
memory, alloc, and so on.
Once you find the name of an undocumented API and the name of the DLL
that exports it, it’s time to look for binaries that use it. Finding an executable
that calls the API will serve two purposes. First, it might shed some additional
light on what the API does. Second, it provides a live sample of how the API is
used and exactly what data it receives as input and what it returns as output.
Finding an example of how a function is used by live code can be invaluable
when trying to learn how to use it.
There are many different approaches for locating APIs and code that uses
them. The traditional approach uses a kernel-mode debugger such as Numega
SoftICE or Microsoft WinDbg. Kernel-mode debuggers make it very easy to
look for calls to a particular function systemwide, even if the function you’re
interested in is not a kernel-mode function. The idea is that you can install sys-
temwide breakpoints that will get hit whenever any process calls some func-
tion. This greatly simplifies the process of finding code that uses a specific
function. You could theoretically do this with a user-mode debugger such as
OllyDbg but it would be far less effective because it would only show you calls
made within the process you’re currently debugging.144 Chapter 5