Calculating The Performance Of A Tool
2
5
Entering edit mode
12.8 years ago
Naga ▴ 450

Usually bioinformatics papers uses Precision, Recall, Accuracy, and MCC for demonstrating the performance of a tool.

What are the other alternative statistical measures, one can use to calculate the performance of a tool (using the same TP, TN, FP and FN) ?

• 3.5k views
ADD COMMENT
12
Entering edit mode
12.8 years ago

The ROCR package for R can calculate lots of performance measures:

Accuracy, error rate, true positive rate, false positive rate, true negative rate, false negative rate, sensitivity, specificity, recall, positive predictive value, negative predictive value, precision, fallout, miss, phi correlation coefficient, Matthews correlation coefficient, mutual information, chi square statistic, odds ratio, lift value, precision/recall F measure, ROC convex hull, area under the ROC curve, precision/recall break-even point, calibration error, mean cross-entropy, root mean squared error, SAR measure, expected cost, explicit cost

ADD COMMENT
0
Entering edit mode

@michaeal: thanks michaeal.. thats a long list. :) just installed the ROCR, will play with it.

ADD REPLY
2
Entering edit mode
12.8 years ago

I would propose uptime in the last x days/weeks/months. If your tool has the ideal scores but has really bad accessibility rates, it still is a bad tool.

It also would be nice to quantify the numbers of issues in a bugtracker and how fast things get fixed. Although it might be difficult to interpret this last metric, since an empty bugtracker could also mean that people are just not interested.

ADD COMMENT
3
Entering edit mode

Did you see the "using the same TP, TN, FP and FN" part of the question? Your answer is more geared towards already-published web tools.

ADD REPLY
0
Entering edit mode

Yes, the question is about getting predictions correct, not other definitions of performance.

ADD REPLY
0
Entering edit mode

Yes, I wanted to know what are all the different performance measure one can use to measure the predictions for publishing.

ADD REPLY

Login before adding your answer.

Traffic: 2243 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6