Question: Minia: Pool allocation failed for 68766984 bytes (kmers alloc), mainbuffer is null?.
0
gravatar for Beuss
3 months ago by
Beuss100
France
Beuss100 wrote:

Hello,

I'm trying to assemble a plant genome (1.1 Gbp) from 38 illumina reads datasets (pair end and mate pair) with a coverage of 150x.

The version of Minia that I use is :

Minia 3, git commit 67ac2ad.

The parameters used are :

minia -in inMinia.txt -out plant_79 -kmer-size 79 -max-memory 500000 -nb-cores 11

The program run on a node with 500 Gb of RAM.

Minia is not able to run the full analysis. It stoped and throw the following errors:

...
      [DSK: Pass 1/1, Step 1: partitioning     ]  43   %   elapsed: 271 min 51 sec   remaining: 360 min 21 sec   cpu: 404.0 %   mem: [7044, 29535, 30387] MB 
      [DSK: Pass 1/1, Step 1: partitioning     ]  44   %   elapsed: 280 min 25 sec   remaining: 356 min 53 sec   cpu: 402.5 %   mem: [3643, 29535, 30387] MB 
      [DSK: Pass 1/1, Step 1: partitioning     ]  45   %   elapsed: 288 min 46 sec   remaining: 352 min 56 sec   cpu: 400.9 %   mem: [1606, 29535, 30387] MB 
      [DSK: Pass 1/1, Step 1: partitioning     ]  46   %   elapsed: 297 min 9  sec   remaining: 348 min 49 sec   cpu: 399.4 %   mem: [ 748, 29535, 30387] MB 
      [DSK: Pass 1/1, Step 2: counting kmers   ]  46.4 %   elapsed: 300 min 6  sec   remaining: 347 min 5  sec   cpu: 399.1 %   mem: [ 127, 29535, 30387] MB EXCEPTION: Pool allocation failed for 76422408 bytes (kmers alloc), mainbuffer is null?. Current usage is 76422424 and capacity is 524288001200
      Pool allocation failed for 68766984 bytes (kmers alloc), mainbuffer is null?. Current usage is 145189416 and capacity is 524288001200
      Pool allocation failed for 71142984 bytes (kmers alloc), mainbuffer is null?. Current usage is 216332408 and capacity is 524288001200
      Pool allocation failed for 73314816 bytes (kmers alloc), mainbuffer is null?. Current usage is 289647232 and capacity is 524288001200
      Pool allocation failed for 61646136 bytes (kmers alloc), mainbuffer is null?. Current usage is 351293384 and capacity is 524288001200
      Pool allocation failed for 81025200 bytes (kmers alloc), mainbuffer is null?. Current usage is 432318592 and capacity is 524288001200
      Pool allocation failed for 80461512 bytes (kmers alloc), mainbuffer is null?. Current usage is 512780120 and capacity is 524288001200
      Pool allocation failed for 66645312 bytes (kmers alloc), mainbuffer is null?. Current usage is 579425440 and capacity is 524288001200
      Pool allocation failed for 74095848 bytes (kmers alloc), mainbuffer is null?. Current usage is 653521304 and capacity is 524288001200
      Pool allocation failed for 75911232 bytes (kmers alloc), mainbuffer is null?. Current usage is 729432544 and capacity is 524288001200
      Pool allocation failed for 64741440 bytes (kmers alloc), mainbuffer is null?. Current usage is 794174000 and capacity is 524288001200

The problem seems to be related with the memory, but, because of the hardware configuration, I'm don't think that the node is lacking of RAM.

Do you any ideas for fixing this ?

Thanks in advance for answers.

minia assembly • 206 views
ADD COMMENTlink modified 3 months ago by guillaume.rizk40 • written 3 months ago by Beuss100
3
gravatar for guillaume.rizk
3 months ago by
France
guillaume.rizk40 wrote:

Hi,

Can you try again with -max-memory 200000 ? I think it tried to allocate the 500 GB you specified and failed since it is exactly the system ram.

ADD COMMENTlink written 3 months ago by guillaume.rizk40

Hi,

I'm think your are right. Before your answer, I launched new assemblies with the -max-memory option set to 0 (no limit I presumed). For now, 3 of the 5 assemblies that I launched finished with no problems. I'm waiting for the results of the two last before confirming the solution.

Thank you for the help.

ADD REPLYlink written 3 months ago by Beuss100
1

I just checked, when you set -max-memory option to 0, it will default to 5 GB max ( or system memory / 2 if system has less than 5 GB). It should run fine, but in your case, if you need to run it again, it should be a little faster with more memory, maybe -max-memory 200000

ADD REPLYlink written 3 months ago by guillaume.rizk40

Thank for the advise ! It will be applied for the next runs.

ADD REPLYlink written 3 months ago by Beuss100
1

Ok the problem is sovled ! Thanks again.

ADD REPLYlink written 3 months ago by Beuss100
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1616 users visited in the last hour