Entering edit mode
6 months ago
Matteo
▴
10
Hi!
I am having issues for the first time with CombineGVCFs. Specifically, it outputs a combined gvcf without chromosome 8 (SUPER_8) even though this is present in the individual gvcfs that I input in the command. There is no error in the log file, the engine just shuts down after chromosome 7, just before chromosome 8. Any idea about what may be going on? Any suggestion will be much appreciated. Thanks in advance!
Matteo
REQUIRED for all errors and issues: a) GATK version used: gatk-4.2.6.1
b) Exact command used:
java -Xmx128g -jar gatk-4.2.6.1/gatk-package-4.2.6.1-local.jar CombineGVCFs \
-R /home/vonholdt/VONHOLDT/reference_genomes/Pogoniulus_pusillus/bPogPus1_combined_assembly.fasta \
-V 230188_pre_bqsr.g.vcf.gz \
-V 990025_pre_bqsr.g.vcf.gz \
-V A241_pre_bqsr.g.vcf.gz \
-V AR69209_pre_bqsr.g.vcf.gz \
-O ../3_Joint_Genotyping/coryphaea1_post_bqsr_gatk.g.vcf.gz
c) Entire program log:
04:39:23.337 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/projects/VONHOLDT/BIN/gatk-4.2.6.1/gatk-package-4.2.6.1-local.jar!/com/intel/gkl/native/libgkl_compression.so
04:39:23.669 INFO CombineGVCFs - ------------------------------------------------------------
04:39:23.670 INFO CombineGVCFs - The Genome Analysis Toolkit (GATK) v4.2.6.1
04:39:23.670 INFO CombineGVCFs - For support and documentation go to https://software.broadinstitute.org/gatk/
04:39:23.670 INFO CombineGVCFs - Executing as ms0553@della-r2c2n9 on Linux v4.18.0-477.27.1.el8_8.x86_64 amd64
04:39:23.670 INFO CombineGVCFs - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_382-b05
04:39:23.670 INFO CombineGVCFs - Start Date/Time: November 1, 2023 4:39:23 AM EDT
04:39:23.670 INFO CombineGVCFs - ------------------------------------------------------------
04:39:23.670 INFO CombineGVCFs - ------------------------------------------------------------
04:39:23.671 INFO CombineGVCFs - HTSJDK Version: 2.24.1
04:39:23.671 INFO CombineGVCFs - Picard Version: 2.27.1
04:39:23.671 INFO CombineGVCFs - Built for Spark Version: 2.4.5
04:39:23.671 INFO CombineGVCFs - HTSJDK Defaults.COMPRESSION_LEVEL : 2
04:39:23.671 INFO CombineGVCFs - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
04:39:23.671 INFO CombineGVCFs - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
04:39:23.671 INFO CombineGVCFs - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
04:39:23.671 INFO CombineGVCFs - Deflater: IntelDeflater
04:39:23.671 INFO CombineGVCFs - Inflater: IntelInflater
04:39:23.671 INFO CombineGVCFs - GCS max retries/reopens: 20
04:39:23.671 INFO CombineGVCFs - Requester pays: disabled
04:39:23.671 INFO CombineGVCFs - Initializing engine
04:39:24.021 INFO FeatureManager - Using codec VCFCodec to read file file:///scratch/gpfs/ms0553/matteo/Pogoniulus_study/snp_calling/4_post_bqsr/2_Combined_VCFs/230188_pre_bqsr.g.vcf.gz
04:39:24.151 INFO FeatureManager - Using codec VCFCodec to read file file:///scratch/gpfs/ms0553/matteo/Pogoniulus_study/snp_calling/4_post_bqsr/2_Combined_VCFs/990025_pre_bqsr.g.vcf.gz
04:39:24.247 INFO FeatureManager - Using codec VCFCodec to read file file:///scratch/gpfs/ms0553/matteo/Pogoniulus_study/snp_calling/4_post_bqsr/2_Combined_VCFs/A241_pre_bqsr.g.vcf.gz
04:39:24.337 INFO FeatureManager - Using codec VCFCodec to read file file:///scratch/gpfs/ms0553/matteo/Pogoniulus_study/snp_calling/4_post_bqsr/2_Combined_VCFs/AR69209_pre_bqsr.g.vcf.gz
04:39:25.050 INFO CombineGVCFs - Done initializing engine
04:39:25.128 INFO ProgressMeter - Starting traversal
04:39:25.128 INFO ProgressMeter - Current Locus Elapsed Minutes Variants Processed Variants/Minute
04:39:25.508 WARN ReferenceConfidenceVariantContextMerger - Detected invalid annotations: When trying to merge variant contexts at location scaffold_261_arrow_ctg1:8487 the annotation MLEAC=[0, 0] was not a numerical value and was ignored
04:39:35.376 INFO ProgressMeter - SUPER_26:1247096 0.2 1285000 7708458.3
04:39:45.131 INFO ProgressMeter - SUPER_26:2866464 0.3 2800000 8398740.2
04:39:55.131 INFO ProgressMeter - SUPER_26:4449520 0.5 4269000 8537146.3
04:40:05.133 INFO ProgressMeter - SUPER_26:6022575 0.7 5836000 8752905.9
04:40:15.134 INFO ProgressMeter - SUPER_26:7535995 0.8 7394000 8871735.4
04:40:25.138 INFO ProgressMeter - SUPER_26:9066749 1.0 8946000 8944509.2
04:40:35.141 INFO ProgressMeter - SUPER_26:10634261 1.2 10497000 8995757.9
04:40:45.147 INFO ProgressMeter - SUPER_26:12218206 1.3 12044000 9030855.2
04:40:55.151 INFO ProgressMeter - SUPER_26:13951281 1.5 13596000 9061684.2
04:41:05.155 INFO ProgressMeter - SUPER_26:15701833 1.7 15155000 9090545.6
04:41:15.156 INFO ProgressMeter - SUPER_26:17423090 1.8 16709000 9111763.5
04:41:25.158 INFO ProgressMeter - SUPER_26:19345502 2.0 18284000 9139791.2
04:41:35.159 INFO ProgressMeter - SUPER_17:485882 2.2 19862000 9164961.9
04:41:45.164 INFO ProgressMeter - SUPER_17:2540828 2.3 21457000 9193493.1
04:41:55.169 INFO ProgressMeter - SUPER_17:4337676 2.5 23023000 9206683.5
04:42:05.170 INFO ProgressMeter - SUPER_17:5801941 2.7 24598000 9221829.3
04:42:15.175 INFO ProgressMeter - SUPER_17:7256963 2.8 26169000 9233564.8
04:42:25.177 INFO ProgressMeter - SUPER_17:8715266 3.0 27740000 9244150.2
04:42:35.180 INFO ProgressMeter - SUPER_17:10153900 3.2 29312000 9253888.4
04:42:45.180 INFO ProgressMeter - SUPER_17:11628308 3.3 30862000 9256193.4
04:42:55.185 INFO ProgressMeter - SUPER_17:13090994 3.5 32414000 9258629.8
04:43:05.189 INFO ProgressMeter - SUPER_17:14589591 3.7 33963000 9260068.8
04:43:15.190 INFO ProgressMeter - SUPER_17:16214217 3.8 35502000 9258895.4
04:43:25.195 INFO ProgressMeter - SUPER_17:17752636 4.0 37040000 9257415.6
04:43:35.199 INFO ProgressMeter - SUPER_17:19245135 4.2 38573000 9254928.6
04:43:45.199 INFO ProgressMeter - SUPER_17:20929710 4.3 40100000 9251319.8
...
05:39:06.267 INFO ProgressMeter - SUPER_7:25969906 59.7 546305000 9153037.6
05:39:16.270 INFO ProgressMeter - SUPER_7:27717550 59.9 547922000 9154558.6
05:39:26.273 INFO ProgressMeter - SUPER_7:29399186 60.0 549486000 9155188.1
05:39:36.274 INFO ProgressMeter - SUPER_7:30952335 60.2 550981000 9154672.8
05:39:46.278 INFO ProgressMeter - SUPER_7:32675838 60.4 552553000 9155428.5
05:39:56.279 INFO ProgressMeter - SUPER_7:34544117 60.5 554182000 9157129.5
05:40:06.281 INFO ProgressMeter - SUPER_7:36497744 60.7 555807000 9158752.7
05:40:16.282 INFO ProgressMeter - SUPER_7:38325768 60.9 557352000 9159054.9
05:40:26.286 INFO ProgressMeter - SUPER_7:40280681 61.0 558840000 9158413.8
05:40:36.288 INFO ProgressMeter - SUPER_7:42426922 61.2 560476000 9160200.0
05:40:42.127 INFO CombineGVCFs - Shutting down engine
[November 1, 2023 5:40:42 AM EDT] org.broadinstitute.hellbender.tools.walkers.CombineGVCFs done. Elapsed time: 61.32 minutes.
Runtime.totalMemory()=4901044224
java.lang.IllegalStateException: The elements of the input Iterators are not sorted according to the comparator htsjdk.variant.variantcontext.VariantContextComparator
at htsjdk.samtools.util.MergingIterator.next(MergingIterator.java:107)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at org.broadinstitute.hellbender.engine.MultiVariantWalker.traverse(MultiVariantWalker.java:136)
at org.broadinstitute.hellbender.engine.MultiVariantWalkerGroupedOnStart.traverse(MultiVariantWalkerGroupedOnStart.java:165)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1085)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
at org.broadinstitute.hellbender.Main.main(Main.java:289)
Thanks Pierre!! That worked!
so validate my answer ( tick on the left )