Maximum size limit for indexing genome with bowtie2 with large index option
0
0
Entering edit mode
3.8 years ago
Sitara • 0

Hello all

I have a fasta file with 100 GB of data and I am trying to create a bowtie2 index. It is showing the following error:

Ran out of memory; automatically trying more memory-economical
parameters. Using parameters --bmax 54 --dcv 4096

Any help would be appreciated. Thanks in advance

genome alignment assembly software error • 2.5k views
ADD COMMENT
0
Entering edit mode

How much memory does the machine have? 100GB is quite large, so you'll need a reasonable amount of memory to even create the index.

ADD REPLY
0
Entering edit mode

I am running on a 256 gb RAM machine with 35 cores

ADD REPLY
0
Entering edit mode

So it's quite possible that there is not enough free memory to build the index with Bowtie2's default settings. You have only ~2.5x as much memory as the size of the input file, and this is likely a shared machine where other processes may be using memory. You can attempt to reduce Bowtie2's indexing size using parameters like --packed and --offrate 7 or --offrate 8 (these latter ones will make mapping slower). Also, it's worth noting that it looks like the message you posted isn't an error. It's a message that lets you know the default parameters are inadequate, and the wrapper is trying to change the parameters to make the index building succeed on your machine. --bmax and --dcv are other parameters that affect the memory required for indexing, and so it's setting them to more conservative values.

ADD REPLY
0
Entering edit mode

Thank you. Will look into it.

ADD REPLY
0
Entering edit mode

Is this 100Gb a single genome? What are you trying to do?

ADD REPLY
0
Entering edit mode

It is a multiple genome fasta file

ADD REPLY
0
Entering edit mode

...and why would you align to such a collection?

ADD REPLY

Login before adding your answer.

Traffic: 1928 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6