Maximum size limit for indexing genome with bowtie2 with large index option
0
0
Entering edit mode
3.7 years ago
Sitara • 0

Hello all

I have a fasta file with 100 GB of data and I am trying to create a bowtie2 index. It is showing the following error:

Ran out of memory; automatically trying more memory-economical
parameters. Using parameters --bmax 54 --dcv 4096


Any help would be appreciated. Thanks in advance

genome alignment assembly software error • 2.4k views
0
Entering edit mode

How much memory does the machine have? 100GB is quite large, so you'll need a reasonable amount of memory to even create the index.

0
Entering edit mode

I am running on a 256 gb RAM machine with 35 cores

0
Entering edit mode

So it's quite possible that there is not enough free memory to build the index with Bowtie2's default settings. You have only ~2.5x as much memory as the size of the input file, and this is likely a shared machine where other processes may be using memory. You can attempt to reduce Bowtie2's indexing size using parameters like --packed and --offrate 7 or --offrate 8 (these latter ones will make mapping slower). Also, it's worth noting that it looks like the message you posted isn't an error. It's a message that lets you know the default parameters are inadequate, and the wrapper is trying to change the parameters to make the index building succeed on your machine. --bmax and --dcv are other parameters that affect the memory required for indexing, and so it's setting them to more conservative values.

0
Entering edit mode

Thank you. Will look into it.

0
Entering edit mode

Is this 100Gb a single genome? What are you trying to do?

0
Entering edit mode

It is a multiple genome fasta file

0
Entering edit mode

...and why would you align to such a collection?