issuhub.com

Reverse Engineering for Beginners

(avery) #1

  • I Code patterns Contents

  • 1 A short introduction to the CPU

    • 1.1 A couple of words about different ISA^4 s



  • 2 The simplest Function

    • 2.1 x86

    • 2.2 ARM.

    • 2.3 MIPS

      • 2.3.1 A note about MIPS instruction/register names





  • 3 Hello, world!

    • 3.1 x86

      • 3.1.1 MSVC

      • 3.1.2 GCC

      • 3.1.3 GCC: AT&T syntax



    • 3.2 x86-64

      • 3.2.1 MSVC—x86-64

      • 3.2.2 GCC—x86-64



    • 3.3 GCC—one more thing.

    • 3.4 ARM.

      • 3.4.1 Non-optimizing Keil 6/2013 (ARM mode)

      • 3.4.2 Non-optimizing Keil 6/2013 (Thumb mode).

      • 3.4.3 Optimizing Xcode 4.6.3 (LLVM) (ARM mode).

      • 3.4.4 Optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode).

      • 3.4.5 ARM64



    • 3.5 MIPS

      • 3.5.1 A word about the “global pointer”

      • 3.5.2 Optimizing GCC.

      • 3.5.3 Non-optimizing GCC.

      • 3.5.4 Role of the stack frame in this example.

      • 3.5.5 Optimizing GCC: load it into GDB.



    • 3.6 Conclusion.

    • 3.7 Exercises.



  • 4 Function prologue and epilogue

    • 4.1 Recursion



  • 5 Stack

    • 5.1 Why does the stack grow backwards?.

    • 5.2 What is the stack used for?

      • 5.2.1 Save the function’s return address.

      • 5.2.2 Passing function arguments.

      • 5.2.3 Local variable storage.

      • 5.2.4 x86: alloca() function.

      • 5.2.5 (Windows) SEH

      • 5.2.6 Buffer overflow protection.

      • 5.2.7 Automatic deallocation of data in stack



    • 5.3 A typical stack layout.

    • 5.4 Noise in stack.

      • 5.4.1 MSVC CONTENTS CONTENTS



    • 5.5 Exercises.



  • 6 printf()with several arguments

    • 6.1 x86

      • 6.1.1 x86: 3 arguments

      • 6.1.2 x64: 8 arguments



    • 6.2 ARM.

      • 6.2.1 ARM: 3 arguments.

      • 6.2.2 ARM: 8 arguments.



    • 6.3 MIPS

      • 6.3.1 3 arguments.

      • 6.3.2 8 arguments.



    • 6.4 Conclusion.

    • 6.5 By the way.



  • 7 scanf()

    • 7.1 Simple example

      • 7.1.1 About pointers

      • 7.1.2 x86

      • 7.1.3 MSVC + OllyDbg

      • 7.1.4 x64

      • 7.1.5 ARM.

      • 7.1.6 MIPS.



    • 7.2 Global variables

      • 7.2.1 MSVC: x86.

      • 7.2.2 MSVC: x86 + OllyDbg

      • 7.2.3 GCC: x86

      • 7.2.4 MSVC: x64.

      • 7.2.5 ARM: Optimizing Keil 6/2013 (Thumb mode)

      • 7.2.6 ARM64

      • 7.2.7 MIPS.



    • 7.3 scanf() result checking

      • 7.3.1 MSVC: x86.

      • 7.3.2 MSVC: x86: IDA.

      • 7.3.3 MSVC: x86 + OllyDbg

      • 7.3.4 MSVC: x86 + Hiew

      • 7.3.5 MSVC: x64.

      • 7.3.6 ARM.

      • 7.3.7 MIPS.

      • 7.3.8 Exercise.



    • 7.4 Exercise



  • 8 Accessing passed arguments

    • 8.1 x86

      • 8.1.1 MSVC

      • 8.1.2 MSVC + OllyDbg

      • 8.1.3 GCC



    • 8.2 x64

      • 8.2.1 MSVC

      • 8.2.2 GCC

      • 8.2.3 GCC: uint64_t instead of int



    • 8.3 ARM.

      • 8.3.1 Non-optimizing Keil 6/2013 (ARM mode)

      • 8.3.2 Optimizing Keil 6/2013 (ARM mode)

      • 8.3.3 Optimizing Keil 6/2013 (Thumb mode).

      • 8.3.4 ARM64



    • 8.4 MIPS



  • 9 More about results returning

    • 9.1 Attempt to use the result of a function returningvoid

    • 9.2 What if we do not use the function result?

    • 9.3 Returning a structure.



  • 10 Pointers CONTENTS CONTENTS

    • 10.1 Global variables example

    • 10.2 Local variables example

    • 10.3 Conclusion.



  • 11 GOTO operator

    • 11.1 Dead code.

    • 11.2 Exercise



  • 12 Conditional jumps

    • 12.1 Simple example

      • 12.1.1 x86

      • 12.1.2 ARM.

      • 12.1.3 MIPS.



    • 12.2 Calculating absolute value.

      • 12.2.1 Optimizing MSVC.

      • 12.2.2 Optimizing Keil 6/2013: Thumb mode

      • 12.2.3 Optimizing Keil 6/2013: ARM mode.

      • 12.2.4 Non-optimizing GCC 4.9 (ARM64)

      • 12.2.5 MIPS.

      • 12.2.6 Branchless version?



    • 12.3 Ternary conditional operator

      • 12.3.1 x86

      • 12.3.2 ARM.

      • 12.3.3 ARM64

      • 12.3.4 MIPS.

      • 12.3.5 Let’s rewrite it in anif/elseway

      • 12.3.6 Conclusion.



    • 12.4 Getting minimal and maximal values.

      • 12.4.1 32-bit.

      • 12.4.2 64-bit.

      • 12.4.3 MIPS.



    • 12.5 Conclusion.

      • 12.5.1 x86

      • 12.5.2 ARM.

      • 12.5.3 MIPS.

      • 12.5.4 Branchless.



    • 12.6 Exercise



  • 13 switch()/case/default

    • 13.1 Small number of cases

      • 13.1.1 x86

      • 13.1.2 ARM: Optimizing Keil 6/2013 (ARM mode).

      • 13.1.3 ARM: Optimizing Keil 6/2013 (Thumb mode)

      • 13.1.4 ARM64: Non-optimizing GCC (Linaro) 4.9

      • 13.1.5 ARM64: Optimizing GCC (Linaro) 4.9

      • 13.1.6 MIPS.

      • 13.1.7 Conclusion.



    • 13.2 A lot of cases

      • 13.2.1 x86

      • 13.2.2 ARM: Optimizing Keil 6/2013 (ARM mode).

      • 13.2.3 ARM: Optimizing Keil 6/2013 (Thumb mode)

      • 13.2.4 MIPS.

      • 13.2.5 Conclusion.



    • 13.3 When there are severalcasestatements in one block

      • 13.3.1 MSVC

      • 13.3.2 GCC

      • 13.3.3 ARM64: Optimizing GCC 4.9.1.



    • 13.4 Fall-through.

      • 13.4.1 MSVC x86

      • 13.4.2 ARM64



    • 13.5 Exercises.

      • 13.5.1 Exercise #1





  • 14 Loops CONTENTS CONTENTS

    • 14.1 Simple example

      • 14.1.1 x86

      • 14.1.2 x86: OllyDbg

      • 14.1.3 x86: tracer.

      • 14.1.4 ARM.

      • 14.1.5 MIPS.

      • 14.1.6 One more thing.



    • 14.2 Memory blocks copying routine

      • 14.2.1 Straight-forward implementation

      • 14.2.2 ARM in ARM mode.

      • 14.2.3 MIPS.

      • 14.2.4 Vectorization



    • 14.3 Conclusion.

    • 14.4 Exercises.



  • 15 Simple C-strings processing

    • 15.1 strlen()

      • 15.1.1 x86

      • 15.1.2 ARM.

      • 15.1.3 MIPS.





  • 16 Replacing arithmetic instructions to other ones

    • 16.1 Multiplication.

      • 16.1.1 Multiplication using addition

      • 16.1.2 Multiplication using shifting.

      • 16.1.3 Multiplication using shifting, subtracting, and adding



    • 16.2 Division

      • 16.2.1 Division using shifts.



    • 16.3 Exercise



  • 17 Floating-point unit

    • 17.1 IEEE 754.

    • 17.2 x86

    • 17.3 ARM, MIPS, x86/x64 SIMD.

    • 17.4 C/C++.

    • 17.5 Simple example

      • 17.5.1 x86

      • 17.5.2 ARM: Optimizing Xcode 4.6.3 (LLVM) (ARM mode)

      • 17.5.3 ARM: Optimizing Keil 6/2013 (Thumb mode)

      • 17.5.4 ARM64: Optimizing GCC (Linaro) 4.9

      • 17.5.5 ARM64: Non-optimizing GCC (Linaro) 4.9

      • 17.5.6 MIPS.



    • 17.6 Passing floating point numbers via arguments.

      • 17.6.1 x86

      • 17.6.2 ARM + Non-optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode)

      • 17.6.3 ARM + Non-optimizing Keil 6/2013 (ARM mode).

      • 17.6.4 ARM64 + Optimizing GCC (Linaro) 4.9.

      • 17.6.5 MIPS.



    • 17.7 Comparison example

      • 17.7.1 x86

      • 17.7.2 ARM.

      • 17.7.3 ARM64

      • 17.7.4 MIPS.



    • 17.8 Stack, calculators and reverse Polish notation

    • 17.9 x64

    • 17.10Exercises.



  • 18 Arrays

    • 18.1 Simple example

      • 18.1.1 x86

      • 18.1.2 ARM.

      • 18.1.3 MIPS.



    • 18.2 Buffer overflow.

      • 18.2.1 Reading outside array bounds.

      • 18.2.2 Writing beyond array bounds CONTENTS CONTENTS



    • 18.3 Buffer overflow protection methods.

      • 18.3.1 Optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode).



    • 18.4 One more word about arrays

    • 18.5 Array of pointers to strings

      • 18.5.1 x64

      • 18.5.2 32-bit ARM.

      • 18.5.3 ARM64

      • 18.5.4 MIPS.

      • 18.5.5 Array overflow



    • 18.6 Multidimensional arrays

      • 18.6.1 Two-dimensional array example

      • 18.6.2 Access two-dimensional array as one-dimensional

      • 18.6.3 Three-dimensional array example

      • 18.6.4 More examples.



    • 18.7 Pack of strings as a two-dimensional array

      • 18.7.1 32-bit ARM.

      • 18.7.2 ARM64

      • 18.7.3 MIPS.

      • 18.7.4 Conclusion.



    • 18.8 Conclusion.

    • 18.9 Exercises.



  • 19 Manipulating specific bit(s)

    • 19.1 Specific bit checking

      • 19.1.1 x86

      • 19.1.2 ARM.



    • 19.2 Setting and clearing specific bits.

      • 19.2.1 x86

      • 19.2.2 ARM + Optimizing Keil 6/2013 (ARM mode).

      • 19.2.3 ARM + Optimizing Keil 6/2013 (Thumb mode)

      • 19.2.4 ARM + Optimizing Xcode 4.6.3 (LLVM) (ARM mode)

      • 19.2.5 ARM: more about theBICinstruction.

      • 19.2.6 ARM64: Optimizing GCC (Linaro) 4.9

      • 19.2.7 ARM64: Non-optimizing GCC (Linaro) 4.9

      • 19.2.8 MIPS.



    • 19.3 Shifts

    • 19.4 Setting and clearing specific bits: FPU^5 example.

      • 19.4.1 A word about theXORoperation.

      • 19.4.2 x86

      • 19.4.3 MIPS.

      • 19.4.4 ARM.



    • 19.5 Counting bits set to 1.

      • 19.5.1 x86

      • 19.5.2 x64

      • 19.5.3 ARM + Optimizing Xcode 4.6.3 (LLVM) (ARM mode)

      • 19.5.4 ARM + Optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode)

      • 19.5.5 ARM64 + Optimizing GCC 4.9

      • 19.5.6 ARM64 + Non-optimizing GCC 4.9

      • 19.5.7 MIPS.



    • 19.6 Conclusion.

      • 19.6.1 Check for specific bit (known at compile stage)

      • 19.6.2 Check for specific bit (specified at runtime).

      • 19.6.3 Set specific bit (known at compile stage)

      • 19.6.4 Set specific bit (specified at runtime).

      • 19.6.5 Clear specific bit (known at compile stage)

      • 19.6.6 Clear specific bit (specified at runtime).



    • 19.7 Exercises.



  • 20 Linear congruential generator

    • 20.1 x86

    • 20.2 x64

    • 20.3 32-bit ARM

    • 20.4 MIPS CONTENTS CONTENTS

      • 20.4.1 MIPS relocations



    • 20.5 Thread-safe version of the example.



  • 21 Structures

    • 21.1 MSVC: SYSTEMTIME example.

      • 21.1.1 OllyDbg.

      • 21.1.2 Replacing the structure with array.



    • 21.2 Let’s allocate space for a structure using malloc()

    • 21.3 UNIX: struct tm.

      • 21.3.1 Linux

      • 21.3.2 ARM.

      • 21.3.3 MIPS.

      • 21.3.4 Structure as a set of values

      • 21.3.5 Structure as an array of 32-bit words

      • 21.3.6 Structure as an array of bytes.



    • 21.4 Fields packing in structure.

      • 21.4.1 x86

      • 21.4.2 ARM.

      • 21.4.3 MIPS.

      • 21.4.4 One more word.



    • 21.5 Nested structures

      • 21.5.1 OllyDbg.



    • 21.6 Bit fields in a structure

      • 21.6.1 CPUID example.

      • 21.6.2 Working with the float type as with a structure.



    • 21.7 Exercises.



  • 22 Unions

    • 22.1 Pseudo-random number generator example.

      • 22.1.1 x86

      • 22.1.2 MIPS.

      • 22.1.3 ARM (ARM mode).



    • 22.2 Calculating machine epsilon

      • 22.2.1 x86

      • 22.2.2 ARM64

      • 22.2.3 MIPS.

      • 22.2.4 Conclusion.



    • 22.3 Fast square root calculation.



  • 23 Pointers to functions

    • 23.1 MSVC

      • 23.1.1 MSVC + OllyDbg

      • 23.1.2 MSVC + tracer.

      • 23.1.3 MSVC + tracer (code coverage)



    • 23.2 GCC

      • 23.2.1 GCC + GDB (with source code).

      • 23.2.2 GCC + GDB (no source code).





  • 24 64-bit values in 32-bit environment

    • 24.1 Returning of 64-bit value

      • 24.1.1 x86

      • 24.1.2 ARM.

      • 24.1.3 MIPS.



    • 24.2 Arguments passing, addition, subtraction

      • 24.2.1 x86

      • 24.2.2 ARM.

      • 24.2.3 MIPS.



    • 24.3 Multiplication, division.

      • 24.3.1 x86

      • 24.3.2 ARM.

      • 24.3.3 MIPS.



    • 24.4 Shifting right

      • 24.4.1 x86

      • 24.4.2 ARM.

      • 24.4.3 MIPS. CONTENTS CONTENTS



    • 24.5 Converting 32-bit value into 64-bit one

      • 24.5.1 x86

      • 24.5.2 ARM.

      • 24.5.3 MIPS.





  • 25 SIMD

    • 25.1 Vectorization

      • 25.1.1 Addition example

      • 25.1.2 Memory copy example



    • 25.2 SIMDstrlen()implementation.



  • 26 64 bits

    • 26.1 x86-64

    • 26.2 ARM.

    • 26.3 Float point numbers.



  • 27 Working with floating point numbers using SIMD

    • 27.1 Simple example

      • 27.1.1 x64

      • 27.1.2 x86



    • 27.2 Passing floating point number via arguments.

    • 27.3 Comparison example

      • 27.3.1 x64

      • 27.3.2 x86



    • 27.4 Calculating machine epsilon: x64 and SIMD

    • 27.5 Pseudo-random number generator example revisited.

    • 27.6 Summary.



  • 28 ARM-specific details

    • 28.1 Number sign (#) before number

    • 28.2 Addressing modes.

    • 28.3 Loading a constant into a register

      • 28.3.1 32-bit ARM.

      • 28.3.2 ARM64



    • 28.4 Relocs in ARM64.



  • 29 MIPS-specific details

    • 29.1 Loading constants into register.

    • 29.2 Further reading about MIPS.



  • II Important fundamentals

  • 30 Signed number representations

  • 31 Endianness

    • 31.1 Big-endian.

    • 31.2 Little-endian

    • 31.3 Example

    • 31.4 Bi-endian

    • 31.5 Converting data



  • 32 Memory

  • 33 CPU

    • 33.1 Branch predictors

    • 33.2 Data dependencies



  • 34 Hash functions

    • 34.1 How do one-way functions work?



  • III Slightly more advanced examples

  • 35 Temperature converting

    • 35.1 Integer values. CONTENTS CONTENTS

      • 35.1.1 Optimizing MSVC 2012 x86.

      • 35.1.2 Optimizing MSVC 2012 x64.



    • 35.2 Floating-point values.



  • 36 Fibonacci numbers

    • 36.1 Example #1

    • 36.2 Example #2

    • 36.3 Summary.



  • 37 CRC32 calculation example

  • 38 Network address calculation example

    • 38.1 calc_network_address().

    • 38.2 form_IP().

    • 38.3 print_as_IP().

    • 38.4 form_netmask() and set_bit()

    • 38.5 Summary.



  • 39 Loops: several iterators

    • 39.1 Three iterators

    • 39.2 Two iterators

    • 39.3 Intel C++ 2011 case.



  • 40 Duff’s device

  • 41 Division by

    • 41.1 x86

    • 41.2 ARM.

      • 41.2.1 Optimizing Xcode 4.6.3 (LLVM) (ARM mode).

      • 41.2.2 Optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode).

      • 41.2.3 Non-optimizing Xcode 4.6.3 (LLVM) and Keil 6/2013



    • 41.3 MIPS

    • 41.4 How it works

      • 41.4.1 More theory



    • 41.5 Getting the divisor.

      • 41.5.1 Variant #1

      • 41.5.2 Variant #2



    • 41.6 Exercise



  • 42 String to number conversion (atoi())

    • 42.1 Simple example

      • 42.1.1 Optimizing MSVC 2013 x64.

      • 42.1.2 Optimizing GCC 4.9.1 x64

      • 42.1.3 Optimizing Keil 6/2013 (ARM mode)

      • 42.1.4 Optimizing Keil 6/2013 (Thumb mode).

      • 42.1.5 Optimizing GCC 4.9.1 ARM64



    • 42.2 A slightly advanced example

      • 42.2.1 Optimizing GCC 4.9.1 x64

      • 42.2.2 Optimizing Keil 6/2013 (ARM mode)



    • 42.3 Exercise



  • 43 Inline functions

    • 43.1 Strings and memory functions

      • 43.1.1 strcmp().

      • 43.1.2 strlen()

      • 43.1.3 strcpy()

      • 43.1.4 memset().

      • 43.1.5 memcpy().

      • 43.1.6 memcmp().

      • 43.1.7 IDA script.





  • 44 C99 restrict

  • 45 Branchlessabs()function

    • 45.1 Optimizing GCC 4.9.1 x64

    • 45.2 Optimizing GCC 4.9 ARM64 CONTENTS CONTENTS



  • 46 Variadic functions

    • 46.1 Computing arithmetic mean.

      • 46.1.1 cdeclcalling conventions.

      • 46.1.2 Register-based calling conventions



    • 46.2vprintf()function case.



  • 47 Strings trimming

    • 47.1 x64: Optimizing MSVC

    • 47.2 x64: Non-optimizing GCC 4.9.1.

    • 47.3 x64: Optimizing GCC 4.9.1.

    • 47.4 ARM64: Non-optimizing GCC (Linaro) 4.9

    • 47.5 ARM64: Optimizing GCC (Linaro) 4.9

    • 47.6 ARM: Optimizing Keil 6/2013 (ARM mode).

    • 47.7 ARM: Optimizing Keil 6/2013 (Thumb mode)

    • 47.8 MIPS



  • 48 toupper() function

    • 48.1 x64

      • 48.1.1 Two comparison operations

      • 48.1.2 One comparison operation.



    • 48.2 ARM.

      • 48.2.1 GCC for ARM64.



    • 48.3 Summary.



  • 49 Incorrectly disassembled code

    • 49.1 Disassembling from an incorrect start (x86).

    • 49.2 How does random noise looks disassembled?



  • 50 Obfuscation

    • 50.1 Text strings

    • 50.2 Executable code

      • 50.2.1 Inserting garbage

      • 50.2.2 Replacing instructions with bloated equivalents

      • 50.2.3 Always executed/never executed code

      • 50.2.4 Making a lot of mess

      • 50.2.5 Using indirect pointers



    • 50.3 Virtual machine / pseudo-code.

    • 50.4 Other things to mention

    • 50.5 Exercise



  • 51 C++

    • 51.1 Classes.

      • 51.1.1 A simple example

      • 51.1.2 Class inheritance.

      • 51.1.3 Encapsulation.

      • 51.1.4 Multiple inheritance.

      • 51.1.5 Virtual methods



    • 51.2 ostream

    • 51.3 References.

    • 51.4 STL

      • 51.4.1 std::string.

      • 51.4.2 std::list

      • 51.4.3 std::vector

      • 51.4.4 std::map and std::set.





  • 52 Negative array indices

  • 53 Windows 16-bit

    • 53.1 Example#1

    • 53.2 Example #2

    • 53.3 Example #3

    • 53.4 Example #4

    • 53.5 Example #5

    • 53.6 Example #6 CONTENTS CONTENTS

      • 53.6.1 Global variables





  • IV Java

  • 54 Java

    • 54.1 Introduction.

    • 54.2 Returning a value

    • 54.3 Simple calculating functions

    • 54.4 JVM^6 memory model

    • 54.5 Simple function calling.

    • 54.6 Calling beep().

    • 54.7 Linear congruential PRNG

    • 54.8 Conditional jumps.

    • 54.9 Passing arguments.

    • 54.10Bitfields

    • 54.11Loops.

    • 54.12switch().

    • 54.13Arrays.

      • 54.13.1Simple example

      • 54.13.2Summing elements of array.

      • 54.13.3The only argument of themain()function is an array too

      • 54.13.4Pre-initialized array of strings.

      • 54.13.5Variadic functions

      • 54.13.6Two-dimensional arrays.

      • 54.13.7Three-dimensional arrays

      • 54.13.8Summary.



    • 54.14Strings

      • 54.14.1First example.

      • 54.14.2Second example



    • 54.15Exceptions.

    • 54.16Classes.

    • 54.17Simple patching

      • 54.17.1 First example.

      • 54.17.2 Second example



    • 54.18Summary.



  • V Finding important/interesting stuff in the code

  • 55 Identification of executable files

    • 55.1 Microsoft Visual C++.

      • 55.1.1 Name mangling.



    • 55.2 GCC

      • 55.2.1 Name mangling.

      • 55.2.2 Cygwin

      • 55.2.3 MinGW



    • 55.3 Intel FORTRAN

    • 55.4 Watcom, OpenWatcom

      • 55.4.1 Name mangling.



    • 55.5 Borland.

      • 55.5.1 Delphi.



    • 55.6 Other known DLLs.



  • 56 Communication with the outer world (win32)

    • 56.1 Often used functions in the Windows API

    • 56.2 tracer: Intercepting all functions in specific module.



  • 57 Strings

    • 57.1 Text strings

      • 57.1.1 C/C++

      • 57.1.2 Borland Delphi

      • 57.1.3 Unicode. CONTENTS CONTENTS

      • 57.1.4 Base64



    • 57.2 Error/debug messages

    • 57.3 Suspicious magic strings.



  • 58 Calls to assert()

  • 59 Constants

    • 59.1 Magic numbers.

      • 59.1.1 Dates

      • 59.1.2 DHCP



    • 59.2 Searching for constants.



  • 60 Finding the right instructions

  • 61 Suspicious code patterns

    • 61.1 XOR instructions.

    • 61.2 Hand-written assembly code



  • 62 Using magic numbers while tracing

  • 63 Other things

    • 63.1 General idea.

    • 63.2 C++

    • 63.3 Some binary file patterns

    • 63.4 Memory “snapshots” comparing

      • 63.4.1 Windows registry.

      • 63.4.2 Blink-comparator.





  • VI OS-specific

  • 64 Arguments passing methods (calling conventions)

    • 64.1 cdecl

    • 64.2 stdcall

      • 64.2.1 Functions with variable number of arguments



    • 64.3 fastcall.

      • 64.3.1 GCC regparm

      • 64.3.2 Watcom/OpenWatcom.



    • 64.4 thiscall.

    • 64.5 x86-64

      • 64.5.1 Windows x64

      • 64.5.2 Linux x64



    • 64.6 Return values offloatanddoubletype

    • 64.7 Modifying arguments.

    • 64.8 Taking a pointer to function argument



  • 65 Thread Local Storage

    • 65.1 Linear congruential generator revisited

      • 65.1.1 Win32.

      • 65.1.2 Linux





  • 66 System calls (syscall-s)

    • 66.1 Linux

    • 66.2 Windows.



  • 67 Linux

    • 67.1 Position-independent code

      • 67.1.1 Windows



    • 67.2 LD_PRELOADhack in Linux



  • 68 Windows NT

    • 68.1 CRT (win32).

    • 68.2 Win32 PE.

      • 68.2.1 Terminology.

      • 68.2.2 Base address

      • 68.2.3 Subsystem. CONTENTS CONTENTS

      • 68.2.4 OS version.

      • 68.2.5 Sections

      • 68.2.6 Relocations (relocs)

      • 68.2.7 Exports and imports.

      • 68.2.8 Resources

      • 68.2.9 .NET.

      • 68.2.10TLS

      • 68.2.11Tools.

      • 68.2.12Further reading.



    • 68.3 Windows SEH.

      • 68.3.1 Let’s forget about MSVC.

      • 68.3.2 Now let’s get back to MSVC

      • 68.3.3 Windows x64

      • 68.3.4 Read more about SEH.



    • 68.4 Windows NT: Critical section



  • VII Tools

  • 69 Disassembler

    • 69.1 IDA



  • 70 Debugger

    • 70.1 OllyDbg

    • 70.2 GDB.

    • 70.3 tracer



  • 71 System calls tracing

    • 71.0.1 strace / dtruss.



  • 72 Decompilers

  • 73 Other tools

  • VIII Examples of real-world RE tasks

  • 74 Task manager practical joke (Windows Vista)

    • 74.1 Using LEA to load values.



  • 75 Color Lines game practical joke

  • 76 Minesweeper (Windows XP)

    • 76.1 Exercises.



  • 77 Hand decompiling + Z3 SMT solver

    • 77.1 Hand decompiling.

    • 77.2 Now let’s use the Z3 SMT solver.



  • 78 Dongles

    • 78.1 Example #1: MacOS Classic and PowerPC

    • 78.2 Example #2: SCO OpenServer.

      • 78.2.1 Decrypting error messages.



    • 78.3 Example #3: MS-DOS.



  • 79 “QR9”: Rubik’s cube inspired amateur crypto-algorithm

  • 80 SAP

    • 80.1 About SAP client network traffic compression

    • 80.2 SAP 6.0 password checking functions.



  • 81 Oracle RDBMS

    • 81.1V$VERSIONtable in the Oracle RDBMS.

    • 81.2X$KSMLRUtable in Oracle RDBMS

    • 81.3V$TIMERtable in Oracle RDBMS



  • 82 Handwritten assembly code CONTENTS CONTENTS

    • 82.1 EICAR test file



  • 83 Demos

    • 83.1 10 PRINT CHR$(205.5+RND(1)); : GOTO

      • 83.1.1 Trixter’s 42 byte version.

      • 83.1.2 My attempt to reduce Trixter’s version: 27 bytes.

      • 83.1.3 Taking random memory garbage as a source of randomness.

      • 83.1.4 Conclusion.



    • 83.2 Mandelbrot set.

      • 83.2.1 Theory

      • 83.2.2 Let’s get back to the demo.

      • 83.2.3 My “fixed” version





  • IX Examples of reversing proprietary file formats

  • 84 Primitive XOR-encryption

    • 84.1 Norton Guide: simplest possible 1-byte XOR encryption.

      • 84.1.1 Entropy.



    • 84.2 Simplest possible 4-byte XOR encryption

      • 84.2.1 Exercise.





  • 85 Millenium game save file

  • 86 Oracle RDBMS: .SYM-files

  • 87 Oracle RDBMS: .MSB-files

    • 87.1 Summary.



  • X Other things

  • 88 npad

  • 89 Executable files patching

    • 89.1 Text strings

    • 89.2 x86 code.



  • 90 Compiler intrinsic

  • 91 Compiler’s anomalies

  • 92 OpenMP

    • 92.1 MSVC

    • 92.2 GCC



  • 93 Itanium

  • 94 8086 memory model

  • 95 Basic blocks reordering

    • 95.1 Profile-guided optimization



  • XI Books/blogs worth reading

  • 96 Books

    • 96.1 Windows.

    • 96.2 C/C++.

    • 96.3 x86 / x86-64

    • 96.4 ARM.

    • 96.5 Cryptography



  • 97 Blogs

    • 97.1 Windows.



  • 98 Other CONTENTS CONTENTS

  • Afterword

  • 99 Questions?

  • Appendix

  • A x86

    • A.1 Terminology.

    • A.2 General purpose registers

      • A.2.1 RAX/EAX/AX/AL.

      • A.2.2 RBX/EBX/BX/BL

      • A.2.3 RCX/ECX/CX/CL.

      • A.2.4 RDX/EDX/DX/DL

      • A.2.5 RSI/ESI/SI/SIL.

      • A.2.6 RDI/EDI/DI/DIL.

      • A.2.7 R8/R8D/R8W/R8L

      • A.2.8 R9/R9D/R9W/R9L

      • A.2.9 R10/R10D/R10W/R10L.

      • A.2.10 R11/R11D/R11W/R11L.

      • A.2.11 R12/R12D/R12W/R12L.

      • A.2.12 R13/R13D/R13W/R13L.

      • A.2.13 R14/R14D/R14W/R14L.

      • A.2.14 R15/R15D/R15W/R15L.

      • A.2.15 RSP/ESP/SP/SPL

      • A.2.16 RBP/EBP/BP/BPL.

      • A.2.17 RIP/EIP/IP

      • A.2.18 CS/DS/ES/SS/FS/GS

      • A.2.19 Flags register



    • A.3 FPU registers

      • A.3.1 Control Word

      • A.3.2 Status Word

      • A.3.3 Tag Word.



    • A.4 SIMD registers

      • A.4.1 MMX registers.

      • A.4.2 SSE and AVX registers.



    • A.5 Debugging registers.

      • A.5.1 DR6

      • A.5.2 DR7



    • A.6 Instructions

      • A.6.1 Prefixes.

      • A.6.2 Most frequently used instructions

      • A.6.3 Less frequently used instructions.

      • A.6.4 FPU instructions

      • A.6.5 Instructions having printable ASCII opcode





  • B ARM

    • B.1 Terminology.

    • B.2 Versions

    • B.3 32-bit ARM (AArch32).

      • B.3.1 General purpose registers

      • B.3.2 Current Program Status Register (CPSR)

      • B.3.3 VFP (floating point) and NEON registers.



    • B.4 64-bit ARM (AArch64).

      • B.4.1 General purpose registers



    • B.5 Instructions

      • B.5.1 Conditional codes table.





  • C MIPS

    • C.1 Registers.

      • C.1.1 General purpose registers GPR

      • C.1.2 Floating-point registers. CONTENTS CONTENTS



    • C.2 Instructions

      • C.2.1 Jump instructions.





  • D Some GCC library functions

  • E Some MSVC library functions

  • F Cheatsheets

    • F.1 IDA

    • F.2 OllyDbg

    • F.3 MSVC

    • F.4 GCC

    • F.5 GDB.



  • Acronyms used

  • Glossary

  • Index

  • Bibliography

Free download pdf
Get our desktop app
Company
Features
Documentation
Resources
© ISSUHUB. 2024. All rights reserved.