Question: Error in running QoRTs after hisat2 alignment
gravatar for bioinfo8
2.6 years ago by
bioinfo8120 wrote:

Hi, I ran QoRTs on multiple bam files and here is the log for one of them:

I tried -Xmx4G, -Xmx8G, -Xmx16G, but still same errors. Please guide. Thanks.

Starting QoRTs v1.2.42 (Compiled Fri Jun  2 12:23:55 EDT 2017)
Starting time: (Mon Oct 16 13:33:39 MEST 2017)
Creating Directory: 1/
Created Log File: 1/QC.eVW6yCQ9UUY7.log
Starting QC
[Time: 2017-10-16 13:33:39] [Mem usage: [85MB / 2058MB]] [Elapsed Time: 00:00:00.0000]
QoRTs is Running in paired-end mode.
QoRTs is Running in any-sorted mode.
NOTE: Function "overlapMatch" requires function "mismatchEngine". Adding "mismatchEngine" to the active function list...
Running functions: CigarOpDistribution, GCDistribution, GeneCalcs, InsertSize, 
        JunctionCalcs, NVC, QualityScoreDistribution, StrandCheck, 
        chromCounts, cigarLocusCounts, mismatchEngine, overlapMatch, 
        readLengthDistro, writeBiotypeCounts, writeClippedNVC, 
        writeDESeq, writeDEXSeq, writeGeneBody, writeGeneCounts, 
        writeGenewiseGeneBody, writeJunctionSeqCounts, 
        writeKnownSplices, writeNovelSplices, writeSpliceExon
Checking first 10000 reads. Checking SAM file for formatting errors...
NOTE: Read length is not consistent.
   In the first 10000 reads, read length varies from 43 to 140 (param maxReadLength=140)
Note that using data that is hard-clipped prior to alignment is NOT recommended, because this makes it difficult (or impossible) to determine the sequencer read-cycle of each nucleotide base. This may obfuscate cycle-specific artifacts, trends, or errors, the detection of which is one of the primary purposes of QoRTs!In addition, hard clipping (whether before or after alignment) removes quality score data, and thus quality score metrics may be misleadingly optimistic. A MUCH preferable method of removing undesired sequence is to replace such sequence with N's, which preserves the quality score and the sequencer cycle information.
   WARNING WARNING WARNING: Read length is not consistent, AND "--maxReadLength" option is not set!
      QoRTs has ATTEMPTED to determine the maximum read length (140).
      It is STRONGLY recommended that you use the --maxReadLength option 
      to set the maximum possible read length, or else errors may occur if/when 
      reads longer than 140 appear.
   Sorting Note: Reads are not sorted by name (This is OK).
   Sorting Note: Reads are sorted by position (This is OK).
Done checking first 10000 reads. WARNINGS FOUND!
SAMRecord Reader Generated. Read length: 140.
[Time: 2017-10-16 13:33:49] [Mem usage: [295MB / 2595MB]] [Elapsed Time: 00:00:10.0813]
Compiling flat feature annotation, internally in memory...
Internal flat feature annotation compiled!
QC Utilities Generated!
[Time: 2017-10-16 13:36:18] [Mem usage: [3221MB / 3894MB]] [Elapsed Time: 00:02:39.0679]
NOTE: Unmatched Read-PAIR-Buffer Size > 100000 [Mem usage:[1665MB / 4062MB]]
    (This is generally not a problem, but if this increases further then OutOfMemoryExceptions
    may occur.
    If memory errors do occur, either increase memory allocation or sort the bam-file by name
    and rerun with the '--nameSorted' option.
    This might also indicate that your dataset contains an unusually large number of
    chimeric read-pairs. Or it could occur simply due to the presence of genomic
    loci with extremly high coverage or complex splicing. It may also indicate a SAM/BAM file that 
    does not adhere to the standard SAM specification.)
NOTE: Unmatched Read-PAIR-Buffer Size > 200000 [Mem usage:[2101MB / 4062MB]]
NOTE: Unmatched Read-PAIR-Buffer Size > 400000 [Mem usage:[3017MB / 4062MB]]
NOTE: Unmatched Read-PAIR-Buffer Size > 800000 [Mem usage:[3604MB / 5GB]]
NOTE: Unmatched Read-PAIR-Buffer Size > 1600000 [Mem usage:[5GB / 7GB]]
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at net.sf.samtools.BinaryTagCodec.readTags(
    at net.sf.samtools.BAMRecord.decodeAttributes(
    at net.sf.samtools.BAMRecord.getAttribute(
    at net.sf.samtools.SAMRecord.isValid(
    at net.sf.samtools.BAMFileReader$BAMFileIterator.advance(
    at net.sf.samtools.BAMFileReader$
    at net.sf.samtools.BAMFileReader$
    at net.sf.samtools.SAMFileReader$
    at net.sf.samtools.SAMFileReader$
    at scala.collection.convert.Wrappers$
    at scala.collection.Iterator$GroupedIterator.takeDestructively(Iterator.scala:930)
    at scala.collection.Iterator$GroupedIterator.go(Iterator.scala:945)
    at scala.collection.Iterator$GroupedIterator.fill(Iterator.scala:985)
    at scala.collection.Iterator$GroupedIterator.hasNext(Iterator.scala:988)
    at internalUtils.stdUtils$$anon$2.hasNext(stdUtils.scala:372)
    at scala.collection.Iterator$JoinIterator.hasNext(Iterator.scala:193)
    at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:406)
    at internalUtils.commonSeqUtils$$anon$5.addNextPairToBuffer(commonSeqUtils.scala:1038)
    at internalUtils.commonSeqUtils$$anon$
    at internalUtils.commonSeqUtils$$anon$
    at internalUtils.stdUtils$IteratorProgressReporter$$anon$
    at scala.collection.Iterator$class.foreach(Iterator.scala:743)
    at internalUtils.stdUtils$IteratorProgressReporter$$anon$5.foreach(stdUtils.scala:487)
    at qcUtils.runAllQC$.runOnSeqFile(runAllQC.scala:1285)
    at qcUtils.runAllQC$.run(runAllQC.scala:939)
    at qcUtils.runAllQC$
    at runner.runner$.main(runner.scala:97)
    at runner.runner.main(runner.scala)
QoRTs done
ADD COMMENTlink modified 2.6 years ago by h.mon29k • written 2.6 years ago by bioinfo8120
gravatar for h.mon
2.6 years ago by
h.mon29k wrote:

QoRTs is extremelly memory-hungry for coordinate-sorted files, despite its manual not making this point clear. If you have lots of memory, you my try increasing the JVM memory with -Xmx, for example -Xmx64g or even higher. Or sort your bam files by name (samtools sort -n -O bam -o file.sort.bam file.bam) and use the --nameSorted QoRTs parameter - for name-sorted bam files, memory usage is indeed low.

Incidentally, QualiMap RNAseq module has the same issue.

ADD COMMENTlink modified 2.6 years ago • written 2.6 years ago by h.mon29k Thanks, I will try and let you know.

ADD REPLYlink written 2.6 years ago by bioinfo8120 I tried with -Xmx64g, but same error. I don't want to sort bam by name as it will take so longer and I have multiple files.

ADD REPLYlink written 2.6 years ago by bioinfo8120

If you don't have more memory, I am afraid you run out of options.

ADD REPLYlink written 2.6 years ago by h.mon29k
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1641 users visited in the last hour