ROC curve for biomarkers

Question: ROC curve for biomarkers

2.8 years ago by

United States

Hello,

For last several days I am trying to draw ROC Curve for my biomarkers study. Unfortunately I did not find any good explanation how it can be done for biomarkers. I would appreciate if you can give me some guidelines.

My current results are from Desq2 and I am not sure how to prepare data input for drawing ROC Curve and also which tool is best to draw it.

Thanks a lot in advance

-- Andrzej

rna-seq next-gen R • 2.0k views

ADD COMMENT • link •

modified 19 months ago by antgomo • 30 • written 2.8 years ago by andrzej.urbanowicz • 0

2.8 years ago by

ddiez • 1.8k

Japan

ddiez • 1.8k wrote:

For plotting I would recommend the ROCR package. See also this and this website. How to use it for biomarker study depends very much on what exactly are you doing.

ADD COMMENT • link written 2.8 years ago by ddiez • 1.8k

Thanks a lot for suggesting tools.

I am doing Differential Expression Analysis using Deseq2 between healthy and unhealthy groups. I have around 20 significantly express genes. I need to check which biomarkers (i.e 3 or more) will give me good AUC (i.e. need value around 0.98).

ADD REPLY • link written 2.8 years ago by andrzej.urbanowicz • 0

Just a tip, you probably want to be careful about how you train and validate (samples left aside to assess performance) your biomarker predictions. An AUC of .98 is generally very high performance for most tasks. Often if you evaluate the performance of your biomarker/predictions based on the same data you used to find them, it will substantially overfit your data and consequently won't generalize to new data (i.e. not reproducible).

ADD REPLY • link written 2.8 years ago by Collin • 680

Of course you are right. I thought about AUC between 0.8 - 0.9.

ADD REPLY • link written 2.8 years ago by andrzej.urbanowicz • 0

I am not an expert at all on ROC but my understanding is that it can be used to determine the performance of a classifier. In you description, it is not clear to me whether you are doing classification. Are you trying to find whether any of the DE genes can be used as biomarkers? That is, whether they can distinguish between healthy and disease? How do you define true positives? (Well, I guess this illustrates my ignorance on the topic).

ADD REPLY • link written 2.8 years ago by ddiez • 1.8k

Yes, you are right. My goal is to find whether any of DE genes can be used as a biomarker for the specific disease (exactly to discriminate between healthy and disease). I want to create prediction curves and to check what AUC value will give me the combination of 3, 5 or 7 chosen genes (how much it will improve) based on the ROC graph. True positive will be the case if the specific biomarker detects the disease (and it is really true in reality).

ADD REPLY • link written 2.8 years ago by andrzej.urbanowicz • 0

It might be that I misunderstood something what you are doing, but shouldn't you construct the model using one dataset (training set ) and use an independent dataset to evaluate the performance to create ROC?

ADD REPLY • link written 2.8 years ago by WouterDeCoster ♦ 40k

Similar posts • Search »