Question: Picard SamFormatConverter gives JVM OutOfMemoryError
0
gravatar for stachele
3.0 years ago by
stachele0
stachele0 wrote:

I am getting an error upon running Picard SamFormatConverter: Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit

Looking in GATK and Biostars threads, this issue seems to come up frequently. The .sam file I start with is 67Gb, which I think is not unreasonably large.

I don't understand the memory requirements for java or where capacity is needed. I believe I have specified near the max memory for java processes by giving the command 20 gigs of memory ("-java -Xmx20g"). I am running all this on an Oracle-VM with Ubuntu 16.04. Windows and Ubuntu both say the computer has ~21 gigs of free RAM

Could there simply be an error in the script that is causing it to think there is not enough memory? Or is there some way to test how much memory is needed? How much memory would likely be enough to run Exome pipelines through Bwa, Picard, GATK, etc?

Thank you very much,

sequencing alignment • 1.1k views
ADD COMMENTlink modified 3.0 years ago by patel.ravip50 • written 3.0 years ago by stachele0
0
gravatar for patel.ravip
3.0 years ago by
patel.ravip50
USA/Philadelphia/iGEM
patel.ravip50 wrote:

Are you sure that you have allocated that much memory for your VM? i.e, you can have a machine with large memory, but if you don't allocate any of it to the VM in the settings, you could have this issue.

Picard is designed to use 2gb of ram for each task, so the -Xmx20g flag for java probably won't have an effect. Instead, if you are sure that you have given enough RAM to your VM, then you can try decreasing the number of MAX_RECORDS_IN_RAM (Picardtools flag) from the default.

EDIT: I misread your original post where you said your VM is Ubuntu and it also says 21GB of ram. I would recommend that you try changing the MAX_RECORDS_IN_RAM option to a smaller number for now, although if the machine and VM have access to 21GB of memory, this is probably not causing the problem. I will also caution that your VM usually shouldn't have access to all the RAM on a machine, as the background host OS still needs to perform tasks to keep the VM running. I have found about 75-80% of the total RAM (on big memory machines; should be less on smaller machines) to be a good upper limit for the VM.

ADD COMMENTlink modified 3.0 years ago • written 3.0 years ago by patel.ravip50
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1395 users visited in the last hour