Data Mining: Practical Machine Learning Tools and Techniques, Second Edition

relation, to -1 when the results are perfectly correlated negatively. Of course,

negative values should not occur for reasonable prediction methods. Correla-

tion is slightly different from the other measures because it is scale independent

in that, if you take a particular set of predictions, the error is unchanged if all

the predictions are multiplied by a constant factor and the actual values are left

unchanged. This factor appears in every term ofSPAin the numerator and in

every term ofSPin the denominator, thus canceling out. (This is not true for

the relative error figures, despite normalization: if you multiply all the predic-

tions by a large constant, then the difference between the predicted and the

actual values will change dramatically, as will the percentage errors.) It is also

different in that good performance leads to a large value of the correlation coef-

ficient, whereas because the other methods measure error, good performance is

indicated by small values.

Which of these measures is appropriate in any given situation is a matter that

can only be determined by studying the application itself. What are we trying

to minimize? What is the cost of different kinds of error? Often it is not easy to

decide. The squared error measures and root squared error measures weigh large

178 CHAPTER 5| CREDIBILITY: EVALUATING WHAT’S BEEN LEARNED

Table 5.8 Performance measures for numeric prediction*.

Performance measure Formula

mean-squared error

root mean-squared error

mean absolute error

relative squared error

root relative squared error

relative absolute error

correlation coefficient

*pare predicted values and aare actual values.

S

pp

n

S

aa

p n

i i

A

= ( - ) i i

=

( - )

ÂÂ

22

11

, and

S

SS

S

ppaa

n

PA

PA

PA

,, where = i(ii- )( - )

Â

1

pa p a

aa a a

nn

n

11

1

-+-

-+-

+

+

...

...

pa p a

aa a a

nn

n

11

22

1

22

( - )+-( )

( - )+-( )

+

+

...

...

pa p a

aa a a

a

n

nn a

n

(^11) i i
22
1
22
( - )+-( ) 1
( - )+-( )

• =


• Â
  ...
  ...
  , where
  pa p a
  n
  11 -++-... nn
  pa p a
  n
  11 nn
  ( - )^22 ++-... ( )
  pa p a
  n
  11 nn
  ( - )^22 +-...+( )