Question: Why do so many biomarker based classifiers claim high accuracy but rarely used in clinical setting?
gravatar for CY
15 months ago by
United States
CY550 wrote:

This may not be technically related to bioinformatics.

Using bioinformatics method, lots of biomarker (methylation; microRNA; mRNA; etc...) based classifiers (for benign / malignant tumor; tumor early detection; tumor subtype; etc) have been published and claim to have good performance. However, it seems that few of them have been used in clinical setting. I can only guess that all these classifiers have some kind of shared issue that forbid them being used in clinical setting. Can anyone please share some opinions on this matter?

biomarker • 362 views
ADD COMMENTlink modified 15 months ago • written 15 months ago by CY550
gravatar for Kevin Blighe
15 months ago by
Kevin Blighe65k
Kevin Blighe65k wrote:

Hey, thanks for posting. Other opinions than mine are very welcome.

Some issues that I have identified over the years:

1, lack of reproducibility of results

This is compounded by poor study design, which itself relates to any number of things:

  • no statistical power
  • imbalanced sample groups
  • failure to control for sources of bias, including age, gender, sampling time, ethnicity, et cetera
  • untested software that contains bugs

I could add here the fact that every instrument that we use has windows of specification in which the instrument is meant to be operated, and they also have error rates. No NGS instrument, for example, can faithfully sequence any sample of DNA - error will always exist.

2, lack of appreciation of biological variability and how to best capture this

This is mainly for expression, ChIP, metabolomic, and proteomic studies

3, slow and costly clinical trials

Clinical trials are very costly and take many years to conduct. Most go nowhere.

4, No 'translational' mechanisms in place

In most cases, there are no official mechanisms / systems in place such that data from a research setting can be readily used in a clinical environment, or, if one exists, validation obviously has to take place, and this may be governed by national and / or international law. If something new is introduced, I notice that it is usually a 'local' change to the health system, i.e., in a hospital unit / department, and not something that is global.


On a side note, I have noticed research change even in the time during which I have been in it (or maybe it is that I have changed). I notice researchers exhibiting greater signs of stress and with less focus on the end goal of the very research that they are conducting. Most do not ponder on what their results could do in terms of improving a health service. Many are also focused intensely on publications and winning the next grant to simply stay in the job. This is a vicious cycle that is ultimately eroding quality research.

I should finally add that I believe more tests will be produced in the next few years. In fact, in certain countries (e.g. Brazil) where there is less regulation, many tests are enrolled straight into practice from research settings. This may prove dangerous, though.


ADD COMMENTlink written 15 months ago by Kevin Blighe65k

Good points. Let's add points 5) apriori probability. If your system detects something with a 99% accuracy (1% FPR) and the general population carries the disease at <= 2%, then your new test is no better than blind chance!

6) no practical workflow. I've seen new techniques touted that detect short term issues, but take days to process or analyze. The test result is obsolete by the time it's ready. Imagine a heart-rate monitor that takes two days to give a result. Pointless clinically, even if it's great fun academically or scientifically.

Any new test needs to be better than what we had before, and affordable, and quick, and error-proof. Machine learning on high-dimensional RNA doesn't satisfy any of that.

ADD REPLYlink written 15 months ago by karl.stamm3.8k

Thanks for the detailed explanation. Really helpful. I am on a project that aims to distinguish lung cancer vs cirrhosis using DNA methylation via liquid biopsy. Can you share some nice review articleS regarding DNA methylation based classifer. I am pretty new in this area.

ADD REPLYlink written 15 months ago by CY550

I am not sure. Probably the best thing is to use a search engine. In this case, I would search for keywords: ncbi review methylation classifier

ADD REPLYlink written 15 months ago by Kevin Blighe65k
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 930 users visited in the last hour