Reverse Engineering for Beginners

CHAPTER 17. FLOATING-POINT UNIT CHAPTER 17. FLOATING-POINT UNIT

So what we get is:

Listing 17.14: Optimizing GCC 4.8.1

fld QWORD PTR [esp+4] ; load "a"

fld QWORD PTR [esp+12] ; load "b"

; ST0=b, ST1=a

fxch st(1)

; ST0=a, ST1=b

; compare "a" and "b"

fucomi st, st(1)

; copy ST1 ("b" here) to ST0 if a<=b

; leave "a" in ST0 otherwise

fcmovbe st, st(1)

; discard value in ST1

fstp st(1)

ret

Hard to guess whyFXCH(swap operands) is here. It’s possible to get rid of it easily by swapping the first twoFLDinstructions

or by replacingFCMOVBE(below or equal) byFCMOVA(above). Probably it’s a compiler inaccuracy.

SoFUCOMIcomparesST(0)(a) andST(1)(b) and then sets some flags in the main CPU.FCMOVBEchecks the flags and

copiesST(1)(bhere at the moment) toST(0)(ahere) ifST0(a)<=ST1(b). Otherwise (a>b), it leavesainST(0).

The lastFSTPleavesST(0)on top of the stack, discarding the contents ofST(1).

Let’s trace this function in GDB:

Listing 17.15: Optimizing GCC 4.8.1 and GDB
1 dennis@ubuntuvm:~/polygon$ gcc -O3 d_max.c -o d_max -fno-inline
2 dennis@ubuntuvm:~/polygon$ gdb d_max
3 GNU gdb (GDB) 7.6.1-ubuntu
4 Copyright (C) 2013 Free Software Foundation, Inc.
5 License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
6 This is free software: you are free to change and redistribute it.
7 There is NO WARRANTY, to the extent permitted by law. Type "show copying"
8 and "show warranty" for details.
9 This GDB was configured as "i686-linux-gnu".
10 For bug reporting instructions, please see:
11 http://www.gnu.org/software/gdb/bugs/...
12 Reading symbols from /home/dennis/polygon/d_max...(no debugging symbols found)...done.
13 (gdb) b d_max
14 Breakpoint 1 at 0x80484a0
15 (gdb) run
16 Starting program: /home/dennis/polygon/d_max
17
18 Breakpoint 1, 0x080484a0 in d_max ()
19 (gdb) ni
20 0x080484a4 in d_max ()
21 (gdb) disas $eip
22 Dump of assembler code for function d_max:
23 0x080484a0 <+0>: fldl 0x4(%esp)
24 => 0x080484a4 <+4>: fldl 0xc(%esp)
25 0x080484a8 <+8>: fxch %st(1)
26 0x080484aa <+10>: fucomi %st(1),%st
27 0x080484ac <+12>: fcmovbe %st(1),%st
28 0x080484ae <+14>: fstp %st(1)
29 0x080484b0 <+16>: ret
30 End of assembler dump.
31 (gdb) ni
32 0x080484a8 in d_max ()
33 (gdb) info float
34 R7: Valid 0x3fff9999999999999800 +1.199999999999999956
35 =>R6: Valid 0x4000d999999999999800 +3.399999999999999911
36 R5: Empty 0x00000000000000000000
37 R4: Empty 0x00000000000000000000
38 R3: Empty 0x00000000000000000000
39 R2: Empty 0x00000000000000000000
40 R1: Empty 0x00000000000000000000
41 R0: Empty 0x00000000000000000000
42