Question: Do I need to test the repeatability and reproducibility of a fixed bioinformatics pipeline for SNV calling
0
gravatar for lzy
5 months ago by
lzy0
lzy0 wrote:

Next-generation sequencing based genetic testing requires proof-of-concept validation with established performance metrics before introducing into the clinical laboratory. The performance metrics evaluated to establish the analytical validity of test results include accuracy, precision, analytical sensitivity, analytical specificity. Precision is typically determined by assessing repeatability and reproducibility. Repeatability (within-run precision) means testing the same sample repeatedly under the same operating conditions (the same people, same time, same place etc.) to evaluate the closeness of agreement between repeated tests. Reproducibility (between-run precision) means the closeness of agreement between the results of measurements when operating conditions are varied(different people, different time etc.). As bioinformatics analysis pipeline is a key component of next-generation sequencing, several guidelines recommended that the bioinformatics analysis pipeline should also be validated. My question is how to evaluate the precision (repeatability and reproducibility) of a bioinformatics analysis pipeline? Or, do I need to evaluate the precision (repeatability and reproducibility) of a bioinformatics analysis pipeline?

sequencing snp next-gen • 193 views
ADD COMMENTlink modified 5 months ago by Nandini720 • written 5 months ago by lzy0

Izy | You might want to make your question clear. As phrased, your post does not contain a question. I think you are wondering what, after clinically important SNVs are discovered/approved, needs to be done to be sure of an individual's status for that SNV. But I am unsure.

ADD REPLYlink written 5 months ago by jnf376940

Thanks, jnf3769. I have revised my question. I just want to know "How to evaluate the precision (repeatability and reproducibility) of a bioinformatics analysis pipeline? Or, do I need to evaluate the precision (repeatability and reproducibility) of a bioinformatics analysis pipeline?"

ADD REPLYlink written 5 months ago by lzy0
2
gravatar for Nandini
5 months ago by
Nandini720
London
Nandini720 wrote:

So, in a clinical setting, we do test the specificity and sensitivity of our bioinformatics pipeline. This can be done in many ways, most common ones are-

1] run the pipeline on 2-3 previously analysed samples and compare results to see if the expected variants are called

2] run the pipeline using, for example, GIAB samples to compare, again, to see if expected variants are called.

More details are here : Guidelines for Validating Next-Generation Sequencing Bioinformatics Pipelines

ADD COMMENTlink modified 5 months ago • written 5 months ago by Nandini720
1

In addition, use a framework that stores all the logs and versions of the tools, databases and references (provenance) you have used. If you have used custom scripts, try to version them

ADD REPLYlink modified 5 months ago • written 5 months ago by cpad011210k

Hi, Nandini. Thanks for your answer. The specificity and sensitivity can be evaluated by running the pipeline on the benchmark data. I want to know do I need to run the pipeline on the same benchmark data 2-3 times to evaluate the repeatability and reproducibility of the pipeline. In my opinion, run the pipeline on the same data will get exactly the same result.

ADD REPLYlink modified 5 months ago • written 5 months ago by lzy0
1

Hi Izy, you need to run the pipeline on different sets of previously analysed data from an older or verified version of the pipeline and see if the variants are called as expected- that is repeatability . As you said, if you run the pipeline 2-3 times on the same data, you will get the same results. Hope this helps.

ADD REPLYlink modified 5 months ago • written 5 months ago by Nandini720
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1145 users visited in the last hour