Question: Bowtie2 run out of memory
0
gravatar for szuszmok
3.8 years ago by
szuszmok40
Hungary
szuszmok40 wrote:

Hi All,

I tried to run bowtie2 with some simulated paired-end sequences, and I got a strange error. I'm using a virtual machine, I got for this VM environment 8 Gb RAM, so I assumed that this could be enough even in case of human data... Besides, before this run I used bowtie2 successfully. Anyone experienced the same before?

My bowtie2 command:

/usr/local/bin/bowtie2 -t -p 4 -D 20 -R 3 -N 0 -L 20 -i S,1,0.50 --rg-id "control_0_D4" --rg "LB:Library_control_0_D4" --rg "PL:Illumina" --rg "PU:PE-1234567890" --rg "SM:control_0_D4" -x /usr/local/share/bcbio/genomes/Hsapiens/GRCh37/bowtie2/human_g1k_v37 -1 ./fastq/control_0_D4_1.fq -2 ./fastq/control_0_D4_2.fq 2>> ./bowtie2/bowtie2.log | samtools view -bhS - | samtools sort - ./bowtie2/control_0_D4;
samtools index ./bowtie2/control_0_D4.bam;

 

and the result of this:

Out of memory allocating plen[] in Ebwt::read() at bt2_io.cpp:272
Overall time: 00:00:00
Error: Encountered exception: 'std::bad_alloc'
Command: /usr/local/bin/../Cellar/

bowtie2/2.2.5/bin/bowtie2-align-s --wrapper basic-0 -t -p 4 -D 20 -R 3 -N 0 -L 20 -i S,1,0.50 --rg-id control_0_D4 --rg LB:Library_control_0_D4 --rg PL:Illumina --rg PU:PE-1234567890 --rg SM:control_0_D4 -x /usr/local/share/bcbio/genomes/Hsapiens/GRCh37/bowtie2/human_g1k_v37 -1 ./fastq/control_0_D4_1.fq -2 ./fastq/control_0_D4_2.fq
(ERR): bowtie2-align exited with value 1
ADD COMMENTlink modified 3.8 years ago • written 3.8 years ago by szuszmok40
1

I'd grab a bigger VM.  8GB is tiny for genomics work.  You can always watch "top" output during execution on a bigger machine and then scale to what you see there.

ADD REPLYlink modified 3.8 years ago • written 3.8 years ago by Sean Davis25k

but before this run, I used this configuration with succes, even when Bowtie2 run for human sequence alignment

ADD REPLYlink modified 3.8 years ago • written 3.8 years ago by szuszmok40

Which version of bowtie2 are you using? There were a couple versions that would rapidly consume RAM sometimes, but not always. I think both Felix Krueger and I ran into problems like that with bisulfite datasets, but it's possible that a similar weird behavior can be invoked with regular data.

ADD REPLYlink written 3.8 years ago by Devon Ryan88k

I'm using the 2.2.5 version

ADD REPLYlink written 3.8 years ago by szuszmok40
1

Have you tracked memory usage as it processes one of the problematic samples? With the cases of this that I'd seen before, memory usage was stable for most of the run and then suddenly spiked exponentially at some point (I don't recall the exact cause, something to do with Ns). It'd be good to know if you see something like that or if you just have a gradual increase in memory consumption.

ADD REPLYlink written 3.8 years ago by Devon Ryan88k

The 'funny' thing is that bowtie2 executed almost like a sec. I think that perhaps my fastq data isn't correct. Thanks for your consideration!

ADD REPLYlink written 3.8 years ago by szuszmok40

That would seem likely then.

ADD REPLYlink written 3.8 years ago by Devon Ryan88k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 622 users visited in the last hour