WEKA software analysis of confusion matrix

Question

*Hi again i am in problem for comparing confusion matrix. Bellow i have provided the two confusion matrix.

  a    b      classified as
   349  58      a  tested_negative
   93   124     b  tested_positive

   a    b    classified as
   346  61    a  tested_negative
   90   127   b  tested_positive
i know that the diagonal of top-left to right but here both that value is same so how can i make decision which one best?*

Answer 1

It actually depends on your specific application. Say you want to minimize number of false positives (because it will cost you a lot to deal with consequences of any false alarm)

In this case, choose the first classifier, because its false positive rate is less then that of the second classifier:

58/(58+124) < 61/(61+127) 0.3186813 < 0.3244681

Take a look here http://en.wikipedia.org/wiki/Accuracy_and_precision

and here: http://en.wikipedia.org/wiki/Sensitivity_and_specificity

If you just want "the best classifier" - you have a problem, since both classifiers have the same accuracy:

a1 = (349+124)/(349+124+58+93) = 0.7580128 a2 = (346+127)/(346+127+61+90) = 0.7580128

So you need to analyze your domain or industry and decide whether you want to:

1) get as little false alarms as possible - then choose classifier with minimum false positive rate;

2) get as little missed cases as possible - then choose classifier with minimum false negative rate;

3) get more hits as possible - then choose classifier with maximum true positive rate;

4) get more correct rejections - then choose classifier with maximum true negative rate.