SAMFormatException: Did not inflate expected amount error
0
0
Entering edit mode
17 months ago

I am facing an error while running GATK Basecalibrator.

[1 July 2021 at 5:00:42 PM IST] org.broadinstitute.hellbender.tools.walkers.bqsr.BaseRecalibrator done. Elapsed time: 35.00 minutes.
Runtime.totalMemory()=791674880
htsjdk.samtools.SAMFormatException: Did not inflate expected amount
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:147)
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:96)
    at htsjdk.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:550)
    at htsjdk.samtools.util.BlockCompressedInputStream.processNextBlock(BlockCompressedInputStream.java:532)
    at htsjdk.samtools.util.BlockCompressedInputStream.nextBlock(BlockCompressedInputStream.java:468)
    at htsjdk.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:458)
    at htsjdk.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:196)
    at htsjdk.samtools.util.BlockCompressedInputStream.read(BlockCompressedInputStream.java:241)
    at htsjdk.tribble.readers.TabixReader.readLine(TabixReader.java:215)
    at htsjdk.tribble.readers.TabixReader.access$300(TabixReader.java:48)
    at htsjdk.tribble.readers.TabixReader$IteratorImpl.next(TabixReader.java:434)
    at htsjdk.tribble.readers.TabixIteratorLineReader.readLine(TabixIteratorLineReader.java:46)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.readNextRecord(TabixFeatureReader.java:170)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.<init>(TabixFeatureReader.java:159)
    at htsjdk.tribble.TabixFeatureReader.query(TabixFeatureReader.java:133)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.refillQueryCache(FeatureDataSource.java:569)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:538)
    at org.broadinstitute.hellbender.engine.FeatureManager.getFeatures(FeatureManager.java:352)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:173)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:125)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:263)
    at org.broadinstitute.hellbender.tools.walkers.bqsr.BaseRecalibrator.apply(BaseRecalibrator.java:189)
    at org.broadinstitute.hellbender.engine.ReadWalker.lambda$traverse$0(ReadWalker.java:96)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
    at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
    at org.broadinstitute.hellbender.engine.ReadWalker.traverse(ReadWalker.java:94)
    at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1058)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
    at org.broadinstitute.hellbender.Main.main(Main.java:289)

What does this mean? Is it related to memory? My system has 16 GB memory, still it is giving me this error again and again. What to do to get rid of this?

WES • 1.9k views
ADD COMMENT
0
Entering edit mode

what was the command line ? what is the output of the command file /path/to/file.gz with all your files with a .gz extension ? what is the output of the command gunzip -t /path/to/file.gz with all your files with a .gz extension ?

ADD REPLY
0
Entering edit mode

with this error, I didn tget any output, so there is no question of output in .gz extension. My command line is: java -jar ../../gatk-4.2.0.0/gatk-package-4.2.0.0-local.jar BaseRecalibrator --input sort_Dup_aligned_reads_gatk.bam --known-sites ../../Common_database/GRCh37_latest_dbSNP_all.vcf.gz --output base_recall.table --reference ../../Common_database/GRCh37_latest_genomic.fna

ADD REPLY
0
Entering edit mode

so there is no question of output in .gz

it is a problem with your inputs

htsjdk.samtools.SAMFormatException: Did not inflate expected amount
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:147)
ADD REPLY
0
Entering edit mode

I am again facing issue while running gatk HaplotypeCaller.

12:49:43.504 INFO  VectorLoglessPairHMM - Time spent in setup for JNI call : 0.16342299700000001
12:49:43.505 INFO  PairHMM - Total compute time in PairHMM computeLogLikelihoods() : 29.143036857000002
12:49:43.505 INFO  SmithWatermanAligner - Total compute time in java Smith-Waterman : 31.03 sec
12:49:43.505 INFO  HaplotypeCaller - Shutting down engine
[16 July 2021 at 12:49:43 PM IST] org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller done. Elapsed time: 3.36 minutes.
Runtime.totalMemory()=1050673152
htsjdk.samtools.SAMFormatException: Did not inflate expected amount
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:147)
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:96)
    at htsjdk.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:550)
    at htsjdk.samtools.util.BlockCompressedInputStream.processNextBlock(BlockCompressedInputStream.java:532)
    at htsjdk.samtools.util.BlockCompressedInputStream.nextBlock(BlockCompressedInputStream.java:468)
    at htsjdk.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:458)
    at htsjdk.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:196)
    at htsjdk.samtools.util.BlockCompressedInputStream.read(BlockCompressedInputStream.java:241)
    at htsjdk.tribble.readers.TabixReader.readLine(TabixReader.java:215)
    at htsjdk.tribble.readers.TabixReader.access$300(TabixReader.java:48)
    at htsjdk.tribble.readers.TabixReader$IteratorImpl.next(TabixReader.java:434)
    at htsjdk.tribble.readers.TabixIteratorLineReader.readLine(TabixIteratorLineReader.java:46)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.readNextRecord(TabixFeatureReader.java:170)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.<init>(TabixFeatureReader.java:159)
    at htsjdk.tribble.TabixFeatureReader.query(TabixFeatureReader.java:133)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.refillQueryCache(FeatureDataSource.java:569)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:538)
    at org.broadinstitute.hellbender.engine.FeatureManager.getFeatures(FeatureManager.java:352)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:173)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:125)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:240)
    at org.broadinstitute.hellbender.tools.walkers.annotator.VariantOverlapAnnotator.annotateRsID(VariantOverlapAnnotator.java:62)
    at org.broadinstitute.hellbender.tools.walkers.annotator.VariantAnnotatorEngine.annotateContext(VariantAnnotatorEngine.java:305)
    at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerGenotypingEngine.makeAnnotatedCall(HaplotypeCallerGenotypingEngine.java:488)
    at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerGenotypingEngine.assignGenotypeLikelihoods(HaplotypeCallerGenotypingEngine.java:224)
    at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerEngine.callRegion(HaplotypeCallerEngine.java:678)
    at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller.apply(HaplotypeCaller.java:273)
    at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.processReadShard(AssemblyRegionWalker.java:200)
    at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.traverse(AssemblyRegionWalker.java:173)
    at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1058)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
    at org.broadinstitute.hellbender.Main.main(Main.java:289)

Earlier also, I am getting the same issue with BaseRecalibrator which was raised in this thread. Can anyone please tell me whats the error about? I re-installed java but still didnt get any output.

ADD REPLY
0
Entering edit mode

So both of these problems remain unresolved?

You should post in the GATK forum for help for a specific response. Someone may still answer here.

ADD REPLY
0
Entering edit mode

GenoMax both the errors are same, related to java. I just want to know what this error is about?

ADD REPLY
0
Entering edit mode

Pierre already told you above that the error is with your input files. Something appears to be wrong with the compression of those files.

ADD REPLY
0
Entering edit mode

... and I asked you for the output of gunzip -t /path/to/file.gz....

ADD REPLY
0
Entering edit mode

I got this: gzip: no_chr_dbsnp.vcf.gz: invalid compressed data--crc error

gzip: no_chr_dbsnp.vcf.gz: invalid compressed data--length error

ADD REPLY
0
Entering edit mode

Hi, Pierre! I encountered the same error in the step of HaplotypeCaller. Which file do you mean? In my case, my output is vcf WITHOUT .gz. But I still got this error.

ADD REPLY
0
Entering edit mode

Hi, smrutimayipanda. Did you solved the problem? I'm working on WGS analysis of a pedigree of three individuals using GATK 4.2.0.0. I encountered exactly the same error in the step of generating gvcf file from bam file.

It was interesting that everything went on well for the first individual; I got gvcf file containing information of all the chromosomes for him. However, for the other two of the individuals, I encountered the error [htsjdk.samtools.SAMFormatException: Did not inflate expected amount] in processing chr6 and chr11, respectively.

You have met this error twice in BaseRecalibrator and HaplotypeCaller. Have you solved the one in BaseRecalibrator?

ADD REPLY
0
Entering edit mode

No I was not able to solve that. I used another pipeline for variant calling.

ADD REPLY
0
Entering edit mode

I contacted with the GATK team. They asked me to try using jdk-inflater and jdk-deflater, but the problem was not solved yet. Which pipeline you've changed to?

ADD REPLY
0
Entering edit mode

freebayes pipeline

ADD REPLY
0
Entering edit mode

Thanks for the information. I think I 'd better shift to that, too.

ADD REPLY

Login before adding your answer.

Traffic: 1517 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6