How to validate a CTD Panel

Question

[jcummins]

Hello everybody, I have a quick question regarding the validation of our paneling system.

So the company that I'm employed by is currently working on a system to conduct a wide variety of "panels" and generate reports of interest for genetic counselors about mutations found in genes that have been linked to various conditions. As of right now we can accept a sequence in FastQ format, and pick out mutations from the resultant VCF created by our pipeline that fall within genes linked to various disorders. I have pieced together a Connective Tissue Disorders panel of Conditions/Genes/Inheritance from PanelApp and various other sources, and now must "validate" this panel.

Honestly in this case I am a little unsure about what would constitute definitive validity. I suppose that ideally I should try running through a sequence or VCF that has been identified to belong to somebody with a condition or conditions such as Marfan Syndrome, LDS, or EDS. However, I have yet to find any such thing, and I am unsure what other ways I can prove that this panel will pick up relevant information besides generating my own dummy VCF.

My apologies if this is a dumb question. I'm a sophomore CS student and the little biology/bioinformatics knowledge that I have has been gained through figuring out this project with 2 other people who don't have a biology/bioinformatics background.

Answer 1

[Kevin Blighe]

So, you're a computer science student and also working with a genetics testing company?

In the US, the major framework for approving new genetic test for use in the clinical setting in the first place is the ACCE, which was developed by Centers for Disease Control and Prevention (CDC) (http://www.cdc.gov/). This framework covers:

• Analytical validity: Ability of the genetic test to accurately measure the genotype/s of interest
• Clinical validity: Ability of the genetic test to detect or infer the presence/absence of the phenotype (disease)
• Clinical utility: Likelihood that the genetic test will lead to improved patient outcome and has addressed financial costs
• Ethical, legal, and social implications

This framework was also adopted in the UK by UKGTN (UK Genetics Testing Network) as their own evaluation for genetic tests; however, the UKGTN built upon the ACCE to form their own framework, called the ‘Gene Dossier’. The Gene Dossier lists the basic information required for genetic test evaluation (in the UK).

Now, further guidelines specifically for NGS genetic tests are then set by the Association for Clinical Genomic Science (ACGS) for UK, and American College of Medical Genetics (ACMG) in the USA. In the USA, further regulation is obviously provide by CLIA, and that is not my area at all.

So, you will have to follow-up on these points to get you started.

From the UK perspective, in order to conduct the validation, one requires a set of samples whose variant calls are known / well-defined by the gold standard. These are then tested by the new testing method (NGS) in order to test sensitivity / specificity.

Finally, when I was developing a NGS pipeline in the UK to introduce NGS genetic testing as part of the live service, one of the panels was a CTD panel, and I worked closely (in the same office) with the head of that section. You may want to reach out to the lab to form a link (or get your supervisor to do so): https://www.sheffieldchildrens.nhs.uk/sdgs/ I also know another CTD academic research group in the UK.

What follows (beneath) are some of my own slides, but these are based on ACGS (UK):

Kevin