How to Interpret FDR-TPR (or FDP-TPR) Curves
0
1
Entering edit mode
4.8 years ago
adam.faranda ▴ 110

I've seen this type of curve now in several papers benchmarking RNA-Sequencing methods. While they look similar to an ROC curve. There are some cases where the curve takes on an unusual trajectory.

TPR-FDP Curve Image from:Van den berge K, Perraudeau F, Soneson C, et al. Observation weights unlock bulk RNA-seq tools for zero inflation and single-cell applications. Genome Biol. 2018;19(1):24.

Intuitively it would make sense if genes were ranked by FDR to calculate a cumulative TPR, however that does not appear to be the case

I'm aware that the iCOBRA package that can be used to generate these curves -- when I look at the fdrtprcurve slot in the cobraplot object from the example data, it looks like FDR can reverse, while TPR increases monotonically

> head(cobplot@fdrtprcurve, 10)
         FDR       TPR NBR       CUTOFF  TP FP   TN  FN TOT_CALLED DIFF NONDIFF
1        NaN 0.0000000   0         -Inf   0  0 1580 819       2399  831    3027
2  0.1606648 0.3646209 361 0.000000e+00 303 58 1522 516       2399  831    3027
3  0.1589041 0.3694344 365 1.110223e-16 307 58 1522 512       2399  831    3027
4  0.1584699 0.3706378 366 2.220446e-16 308 58 1522 511       2399  831    3027
5  0.1580381 0.3718412 367 3.330669e-16 309 58 1522 510       2399  831    3027
6  0.1576087 0.3730445 368 5.551115e-16 310 58 1522 509       2399  831    3027
7  0.1594595 0.3742479 370 7.771561e-16 311 59 1521 508       2399  831    3027
8  0.1590296 0.3754513 371 9.992007e-16 312 59 1521 507       2399  831    3027
9  0.1581769 0.3778580 373 1.110223e-15 314 59 1521 505       2399  831    3027
10 0.1604278 0.3778580 374 1.221245e-15 314 60 1520 505       2399  831    3027

An explanation of how these values are calculated, and the plots are interpreted would be very helpful. Thanks also to ATpoint for fixing my image insert.

RNA-Seq • 1.8k views
ADD COMMENT
1
Entering edit mode

It is not a False Discovery Rate, it is False Discovery Proportion - similar but different concept

ADD REPLY
1
Entering edit mode

The unusual trajectory is explained at the legend of the figure:

The behavior in the lower half of the curve for MAST in (b) is due to a smooth increase in true positives with an identical number of false positives over a range of low FDR cut-offs.

ADD REPLY
0
Entering edit mode

Link to the paper, please.

ADD REPLY

Login before adding your answer.

Traffic: 1244 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6