calculate accuracy and precision of confusion matrix in R

Question

Is there any tool / R package available to calculate accuracy and precision of confusion matrix in R ?

The formula and data structure are here

Solution 1

yes, you can calculate Accuracy and precision in R with confusion matrix. It uses Caret package.

Here is the example :

lvs <- c("normal", "abnormal")
truth <- factor(rep(lvs, times = c(86, 258)),
                levels = rev(lvs))
pred <- factor(
               c(
                 rep(lvs, times = c(54, 32)),
                 rep(lvs, times = c(27, 231))),               
               levels = rev(lvs))

xtab <- table(pred, truth)
# load Caret package for computing Confusion matrix
library(caret) 
confusionMatrix(xtab)

And Confusion Matrix for xtab would be like this :

Confusion Matrix and Statistics

          truth
pred       abnormal normal
  abnormal      231     32
  normal         27     54

               Accuracy : 0.8285
                 95% CI : (0.7844, 0.8668)
    No Information Rate : 0.75
    P-Value [Acc > NIR] : 0.0003097

                  Kappa : 0.5336
 Mcnemar's Test P-Value : 0.6025370

            Sensitivity : 0.8953
            Specificity : 0.6279
         Pos Pred Value : 0.8783
         Neg Pred Value : 0.6667
             Prevalence : 0.7500
         Detection Rate : 0.6715
   Detection Prevalence : 0.7645

       'Positive' Class : abnormal

So here is everything, that you want.

Solution 2

@Harsh Trivedi

byClass allows you to pull out the precision and recall from the summary. PPV is precision. Sensitivity is recall. https://en.wikipedia.org/wiki/Precision_and_recall

library(caret)

result <- confusionMatrix(prediction, truth)
precision <- result$byClass['Pos Pred Value']    
recall <- result$byClass['Sensitivity']

I imagine you want to pull out the precision and recall to calculate the f-measure so here it goes.

f_measure <- 2 * ((precision * recall) / (precision + recall))

I also found this handy online calculator for sanity check. http://www.marcovanetti.com/pages/cfmatrix/?noc=2

-bg

Comments

[Harsh Trivedi]

how to programatically find precision and recall after results of confusionMatrix(xtab) are got?

[Rupesh]

Thanks Nishu. Just one extra information I want to add. confusionMatrix(xtab) has dependency on "e1071" package, so installation of this package may be required.

[Nishu Tayal]

@Rupesh: yeah, that's required.