Why do so many biomarker based classifiers claim high accuracy but rarely used in clinical setting?
1
5
Entering edit mode
3.3 years ago
CY ▴ 670

This may not be technically related to bioinformatics.

Using bioinformatics method, lots of biomarker (methylation; microRNA; mRNA; etc...) based classifiers (for benign / malignant tumor; tumor early detection; tumor subtype; etc) have been published and claim to have good performance. However, it seems that few of them have been used in clinical setting. I can only guess that all these classifiers have some kind of shared issue that forbid them being used in clinical setting. Can anyone please share some opinions on this matter?

biomarker • 982 views
10
Entering edit mode
3.3 years ago

Hey, thanks for posting. Other opinions than mine are very welcome.

Some issues that I have identified over the years:

## 1, lack of reproducibility of results

This is compounded by poor study design, which itself relates to any number of things:

• no statistical power
• imbalanced sample groups
• failure to control for sources of bias, including age, gender, sampling time, ethnicity, et cetera
• untested software that contains bugs

I could add here the fact that every instrument that we use has windows of specification in which the instrument is meant to be operated, and they also have error rates. No NGS instrument, for example, can faithfully sequence any sample of DNA - error will always exist.

## 2, lack of appreciation of biological variability and how to best capture this

This is mainly for expression, ChIP, metabolomic, and proteomic studies

## 3, slow and costly clinical trials

Clinical trials are very costly and take many years to conduct. Most go nowhere.

## 4, No 'translational' mechanisms in place

In most cases, there are no official mechanisms / systems in place such that data from a research setting can be readily used in a clinical environment, or, if one exists, validation obviously has to take place, and this may be governed by national and / or international law. If something new is introduced, I notice that it is usually a 'local' change to the health system, i.e., in a hospital unit / department, and not something that is global.

## ------------------------------

On a side note, I have noticed research change even in the time during which I have been in it (or maybe it is that I have changed). I notice researchers exhibiting greater signs of stress and with less focus on the end goal of the very research that they are conducting. Most do not ponder on what their results could do in terms of improving a health service. Many are also focused intensely on publications and winning the next grant to simply stay in the job. This is a vicious cycle that is ultimately eroding quality research.

I should finally add that I believe more tests will be produced in the next few years. In fact, in certain countries (e.g. Brazil) where there is less regulation, many tests are enrolled straight into practice from research settings. This may prove dangerous, though.

Kevin

3
Entering edit mode

Good points. Let's add points 5) apriori probability. If your system detects something with a 99% accuracy (1% FPR) and the general population carries the disease at <= 2%, then your new test is no better than blind chance!

6) no practical workflow. I've seen new techniques touted that detect short term issues, but take days to process or analyze. The test result is obsolete by the time it's ready. Imagine a heart-rate monitor that takes two days to give a result. Pointless clinically, even if it's great fun academically or scientifically.

Any new test needs to be better than what we had before, and affordable, and quick, and error-proof. Machine learning on high-dimensional RNA doesn't satisfy any of that.

0
Entering edit mode

Thanks for the detailed explanation. Really helpful. I am on a project that aims to distinguish lung cancer vs cirrhosis using DNA methylation via liquid biopsy. Can you share some nice review articleS regarding DNA methylation based classifer. I am pretty new in this area.

0
Entering edit mode

I am not sure. Probably the best thing is to use a search engine. In this case, I would search for keywords: ncbi review methylation classifier