Bowtie2 run out of memory
0
0
Entering edit mode
7.9 years ago
anon ▴ 50

Hi All,

I tried to run bowtie2 with some simulated paired-end sequences, and I got a strange error. I'm using a virtual machine, I got for this VM environment 8 Gb RAM, so I assumed that this could be enough even in case of human data... Besides, before this run I used bowtie2 successfully. Anyone experienced the same before?

My bowtie2 command:

/usr/local/bin/bowtie2 \
-t \
-p 4 \
-D 20 \
-R 3 \
-N 0 \
-L 20 \
-i S,1,0.50 \
--rg-id "control_0_D4" \
--rg "LB:Library_control_0_D4" \
--rg "PL:Illumina" \
--rg "PU:PE-1234567890" \
--rg "SM:control_0_D4" \
-x /usr/local/share/bcbio/genomes/Hsapiens/GRCh37/bowtie2/human_g1k_v37 \
-1 ./fastq/control_0_D4_1.fq \
-2 ./fastq/control_0_D4_2.fq 2 >> ./bowtie2/bowtie2.log |\
samtools view -bhS - | \
samtools sort - ./bowtie2/control_0_D4;
samtools index ./bowtie2/control_0_D4.bam


and the result of this:

Out of memory allocating plen[] in Ebwt::read() at bt2_io.cpp:272
Overall time: 00:00:00
Command: /usr/local/bin/../Cellar/

bowtie2/2.2.5/bin/bowtie2-align-s --wrapper basic-0 -t -p 4 -D 20 -R 3 -N 0 -L 20 -i S,1,0.50 --rg-id control_0_D4 --rg LB:Library_control_0_D4 --rg PL:Illumina --rg PU:PE-1234567890 --rg SM:control_0_D4 -x /usr/local/share/bcbio/genomes/Hsapiens/GRCh37/bowtie2/human_g1k_v37 -1 ./fastq/control_0_D4_1.fq -2 ./fastq/control_0_D4_2.fq
(ERR): bowtie2-align exited with value 1

software-error alignment bowtie2 • 6.7k views
1
Entering edit mode

I'd grab a bigger VM. 8GB is tiny for genomics work. You can always watch "top" output during execution on a bigger machine and then scale to what you see there.

0
Entering edit mode

but before this run, I used this configuration with succes, even when Bowtie2 run for human sequence alignment

0
Entering edit mode

Which version of bowtie2 are you using? There were a couple versions that would rapidly consume RAM sometimes, but not always. I think both Felix Krueger and I ran into problems like that with bisulfite datasets, but it's possible that a similar weird behavior can be invoked with regular data.

0
Entering edit mode

I'm using the 2.2.5 version

1
Entering edit mode

Have you tracked memory usage as it processes one of the problematic samples? With the cases of this that I'd seen before, memory usage was stable for most of the run and then suddenly spiked exponentially at some point (I don't recall the exact cause, something to do with Ns). It'd be good to know if you see something like that or if you just have a gradual increase in memory consumption.

0
Entering edit mode

The 'funny' thing is that bowtie2 executed almost like a sec. I think that perhaps my fastq data isn't correct. Thanks for your consideration!

0
Entering edit mode

That would seem likely then.