Question

Memory Usage Of Abyss Assembler

3

Entering edit mode

12.4 years ago

Bioscientist ★ 1.7k

I don't know if anyone here use ABYSS for assembly. (or any other assemblers) I have around 45GB fastq file for assembly onto human reference genome using ABYSS. I'm now only running on single processors, and used all 48GB memory on the computer;(48GB is the largest RAM for computer at my institution), and seems it just takes forever to finish.

I'm curious, how much RAM we need to run ABYSS for 45GB input? I googled for a while and found someone said they used like 256GB RAM to run.

And the most advanced machine here is 48GB total RAM, with 24 processors. That means you can use 20+ processors to speed up, but RAM limit is still 48GB. So, suppose we really need 256GB RAM, then even if I run in parallel processors, it still won't work, right?

I used to only mostly play with mapping; for mapping I can split input file if it's too large; but for assembly....

assembly • 5.8k views

ADD COMMENT • link updated 12.4 years ago by Manu Prestat 4.1k • written 12.4 years ago by Bioscientist ★ 1.7k

0

Entering edit mode

Seems like you should be able to reduce memory footprint by using a smaller value of k for the de Bruijn graph (at the cost of a worse assembly). Have you tried this?

ADD REPLY • link 12.4 years ago by Qdjm 1.9k

0

Entering edit mode

I tried k=25,k=35,k=45; (k=25 is recommended). But seems they are equally memory-costly.

ADD REPLY • link 12.4 years ago by Bioscientist ★ 1.7k

score 2 · Answer 1 · 2011-12-15

2

Entering edit mode

12.4 years ago

Zev.Kronenberg 12k

We do a fair about of assembly in our lab. What we found is if you have 50x coverage SOAPdenovo takes at least 250gig and ABYSS takes even more. For assembling we are upgrading to a 1TB (RAM) machine.

ADD COMMENT • link 12.4 years ago by Zev.Kronenberg 12k

2

Entering edit mode

If memory is a concern, please check out the recent SGA paper. <64GB for 30X human.

ADD REPLY • link 12.4 years ago by lh3 33k

0

Entering edit mode

Can you confirm me that if speed is concern, SGA is not a good choice in assembling large genome?

ADD REPLY • link 7.2 years ago by saranpons3 ▴ 70

0

Entering edit mode

Sounds interesting I will have to check it out.

ADD REPLY • link 12.4 years ago by Zev.Kronenberg 12k

score 0 · Answer 2 · 2011-12-15

Take a look at CLC Assembly Cell from CLC bio. It is supposed to use half RAM comparing to SOAP de novo assembly for instance, and to give better quality of assemblies. The difference is that this one is not free, but I heard a lot of good things about it. And, it may be cheaper than 256 GB of extra RAM...

I have not tried it yet.