Question: Maximum size limit for indexing genome with bowtie2 with large index option
0
gravatar for Sitara
8 weeks ago by
Sitara0
Sitara0 wrote:

Hello all

I have a fasta file with 100 GB of data and I am trying to create a bowtie2 index. It is showing the following error:

Ran out of memory; automatically trying more memory-economical
parameters. Using parameters --bmax 54 --dcv 4096

Any help would be appreciated. Thanks in advance

ADD COMMENTlink modified 7 days ago by Biostar ♦♦ 20 • written 8 weeks ago by Sitara0

How much memory does the machine have? 100GB is quite large, so you'll need a reasonable amount of memory to even create the index.

ADD REPLYlink written 8 weeks ago by Rob2.8k

I am running on a 256 gb RAM machine with 35 cores

ADD REPLYlink written 8 weeks ago by Sitara0

So it's quite possible that there is not enough free memory to build the index with Bowtie2's default settings. You have only ~2.5x as much memory as the size of the input file, and this is likely a shared machine where other processes may be using memory. You can attempt to reduce Bowtie2's indexing size using parameters like --packed and --offrate 7 or --offrate 8 (these latter ones will make mapping slower). Also, it's worth noting that it looks like the message you posted isn't an error. It's a message that lets you know the default parameters are inadequate, and the wrapper is trying to change the parameters to make the index building succeed on your machine. --bmax and --dcv are other parameters that affect the memory required for indexing, and so it's setting them to more conservative values.

ADD REPLYlink written 8 weeks ago by Rob2.8k

Thank you. Will look into it.

ADD REPLYlink written 8 weeks ago by Sitara0

Is this 100Gb a single genome? What are you trying to do?

ADD REPLYlink written 8 weeks ago by h.mon21k

It is a multiple genome fasta file

ADD REPLYlink written 8 weeks ago by Sitara0

...and why would you align to such a collection?

ADD REPLYlink written 8 weeks ago by ATpoint9.1k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1889 users visited in the last hour