Question: Picard Markduplicates Error: Requested Array Size Exceeds Vm Limit
0
gravatar for xiaoyanli82
5.8 years ago by
xiaoyanli820 wrote:

Hi, everyone,

I am working with wheat exome data, which are Illumina PE 100 reads. I analysed the data followed the GATK pipeline. Read were aligned to references using BWA and the BAM files were sorted by samtools. But when I tried to mark PCR dupication using Picard, I got error:

 java -Xms16g -Xmx256g -jar MarkDuplicates.jar I=IN.bam O=OUT.DeDup.bam METRICS_FILE=OUT.dedup REMOVE_DUPLICATES=false MAX_FILE_HANDLES_FOR_READ_ENDS_MAP=1000 VALIDATION_STRINGENCY=LENIENT

Error :Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit

I also tried to other options:

java -Xmx160g -jar MarkDuplicates.jar I=IN.bam O=OUT.DeDup.bam METRICS_FILE=OUT.dedup REMOVE_DUPLICATES=false MAX_FILE_HANDLES_FOR_READ_ENDS_MAP=1000 VALIDATION_STRINGENCY=LENIENT

Error:java.lang.OutOfMemoryError: Java heap space

The BAM file is 4G, java version is 1.7.0_11, picard is version 1.92 What can I do to solve this problem?

Thank you !

Xiaoyan

picard markduplicates • 6.7k views
ADD COMMENTlink modified 5.8 years ago by Jordan1.1k • written 5.8 years ago by xiaoyanli820
1
gravatar for Nandini
5.8 years ago by
Nandini760
London
Nandini760 wrote:

I has the same problem for human genomes and adding "MAX_RECORDS_IN_RAM=5000000" helped me. Maybe you can try that, here's the code that I use

 java -Xmx16g -jar MarkDuplicates.jar    \
    I=IN.bam \
    O=OUT.bam \
    METRICS_FILE=dupmetrics.txt \
    REMOVE_DUPLICATES=true \
    MAX_RECORDS_IN_RAM=5000000 \
    ASSUME_SORTED=true \
    VALIDATION_STRINGENCY=SILENT \
    TMP_DIR=$TMPDIR \
    CREATE_INDEX=true \
    OPTICAL_DUPLICATE_PIXEL_DISTANCE=10
ADD COMMENTlink written 5.8 years ago by Nandini760

The error is due to Java heap space. The user tells the tool that it has 256 Gb of RAM that I don't think is normal. So I dont think "MAX_RECORDS_IN_RAM" is an issue.

ADD REPLYlink written 5.8 years ago by Ashutosh Pandey11k

I tried again following your suggestion, the problem was fixed, thank you very much!

ADD REPLYlink written 5.8 years ago by xiaoyanli820
0
gravatar for Ashutosh Pandey
5.8 years ago by
Philadelphia
Ashutosh Pandey11k wrote:

Does your computer has 256 Gb RAM ? because you specified it on your command line -Xmx256g . If your bam file is 4 Gb, then I wont even care about changing that parameter and go with the default one.

ADD COMMENTlink written 5.8 years ago by Ashutosh Pandey11k

Yes, the system has 1024g RAM.

ADD REPLYlink written 5.8 years ago by xiaoyanli820
0
gravatar for Jordan
5.8 years ago by
Jordan1.1k
Pittsburgh
Jordan1.1k wrote:

I'm just going to guess here. Did you try increasing your PermGen space? If your perm gen space is low, no matter how much heap space you increase, it does not matter.

Try this:

-XX:MaxPermSize=1g

Or how much ever you think is necessary.

ADD COMMENTlink modified 5.8 years ago • written 5.8 years ago by Jordan1.1k

It should be "-XX:MaxPermSize=1g", but I don't think this is an issue. Allocating (trying) so much memory is the one, IMO.

ADD REPLYlink modified 5.8 years ago • written 5.8 years ago by Pavel Senin1.9k

Oh yea. Sorry about. Fixed it. And it could be like you said too. But I'm curious to know how much RAM his system actually has!

ADD REPLYlink written 5.8 years ago by Jordan1.1k

The linux system has 1024g RAM

ADD REPLYlink written 5.8 years ago by xiaoyanli820

Sounds good then. What is OS and JVM implementation you are running? And what would be the output if you specify -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+PrintGCDetails -XX:+PrintGCTimeStamps?

ADD REPLYlink modified 5.8 years ago • written 5.8 years ago by Pavel Senin1.9k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1182 users visited in the last hour