Question: memory consumption of bwa-mem
1
gravatar for Lars
3.5 years ago by
Lars750
Cologne
Lars750 wrote:

Hi all, I am working with Human amplicon data and have been using bwa mem (0.7.15) up to now. Unfortunately, on some datasets the memory consumption exceeds the 8GB RAM I have at spare.

Can someone tell me why this is the case?

I have already tried to modify the parameter (-c or -t) but none of them made it able to run it. Could it be caused by parts of the data aligning to repeat regions/low complexity regions?

If I cannot find a solution to it, I will switch to an alternative aligner that does not need that really high amount of RAM.

memory mapping ram bwa-mem • 5.2k views
ADD COMMENTlink modified 3.5 years ago by d-cameron2.2k • written 3.5 years ago by Lars750
2

8 GB is not really high amount of RAM. Don't most bottom of the line laptops ship with 8 GB these days? Hell, my phone has 6 GB RAM..

ADD REPLYlink modified 3.5 years ago • written 3.5 years ago by 5heikki9.0k
1

Thank you for this information about your phone with 6GB RAM. I cannot see any use in this comment.

I have a HPC with 2 TB RAM and 32 cores. But this amplicon project has to run on a laptop with exact 8GB RAM. It looks like bwa-mem is the wrong tool then. Thank you anyway.

ADD REPLYlink written 3.5 years ago by Lars750

You should specify what you are aligning against. If it is not human genome then you could take a look at BBMap.

ADD REPLYlink written 3.5 years ago by genomax89k

I align against the human genome.

ADD REPLYlink written 3.5 years ago by Lars750

But this amplicon project has to run on a laptop with exact 8GB RAM.

Then you have to consider aligning against a subset of the human genome. Perhaps just the regions you are amplifying.

ADD REPLYlink written 3.5 years ago by genomax89k

That is not an option. It has to be the complete human genome and on exactly this laptop. My question: Is it possible with bwa-mem, by using some specific parameters?

ADD REPLYlink written 3.5 years ago by Lars750

Did you see Heng Li's (author of bwa) answer that I had linked below?

ADD REPLYlink written 3.5 years ago by genomax89k

Yes, I know this comment of Heng Li and I tried it, but it didn't work.

ADD REPLYlink written 3.5 years ago by Lars750

Phone having 6 GB RAM goes to show how exhausting 8 GB of RAM is not "incredible high memory consumption" as in the original title of your post..

ADD REPLYlink written 3.5 years ago by 5heikki9.0k

I already changed it. Wanted to attract people. ;)

ADD REPLYlink written 3.5 years ago by Lars750
2

If you are using human genome to align against then ~6 G is minimally required by bwa for one thread. Other aligners actually will need more than 8 G.

ADD REPLYlink modified 3.5 years ago • written 3.5 years ago by genomax89k

Invest in your future and get a better machine.

ADD REPLYlink written 3.5 years ago by David Langenberger9.5k
0
gravatar for d-cameron
3.5 years ago by
d-cameron2.2k
Australia
d-cameron2.2k wrote:

Close to 8GB is a fairly typical index size for a human reference gnome [1]. If poor performance is acceptable, or the memory usage is only just over 8GB, you may wish to considering increasing and/or adding additional swap space to allow bwa to run.

Be aware that amplicon sequencing has artefacts not present in WGS sequencing that your pipeline may need to address [2,3].

[1] https://doi.org/10.1093/bioinformatics/bts605

[2] http://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-15-1073

[3] https://academic.oup.com/nar/article/43/6/e37/2453415/Insight-into-biases-and-sequencing-errors-for

ADD COMMENTlink modified 3.5 years ago • written 3.5 years ago by d-cameron2.2k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 2045 users visited in the last hour