Question: NGS compute infrastructure
0
gravatar for Cosmo
19 months ago by
Cosmo10
Cosmo10 wrote:

Hello,

I am a new PI working in the area of computational genomics, who is looking at setting up a compute infrastructure for my lab. There are two main computational tasks that the lab will be performing: 1. simulations (100-1000s of jobs each with a run time of a few seconds up to a few minutes) and 2. genome alignments, SNP calling, etc (only a few jobs but with higher RAM requirements). As such, I am looking into two different options: one system with a large amount of RAM but few CPUs and one with many CPUs with less required RAM or alternatively a solution where RAM can be temporarily shared (ideally with a RAID5 or RAID6). I would greatly appreciate if someone could share their experience with different compute architectures with me (as well as which companies they can recommend).

Thanks!

sequencing alignment next-gen • 669 views
ADD COMMENTlink written 19 months ago by Cosmo10

Make sure to have sufficient I/O capacity to really make full use of CPU and RAM. The best cluster makes no sense if the I/O bottleneck kills all the performance and permits to use multithreading effectively.

ADD REPLYlink written 19 months ago by ATpoint14k

I have no experience with this, but if you have a hard time estimating your needs you could also look at more flexible cloud-based solutions for which you pay what you use/need when you need it. Perhaps others have a different opinion about this.

ADD REPLYlink written 19 months ago by WouterDeCoster37k

From your post it seems that you may be conflating RAM with storage space.

RAM cannot be shared via a RAID - this latter word stands for "redundant array of independent disks" so they are hard drive storage systems no computer memory.

  • RAM - is the computer memory that programs can use when they run and are in tens into the hundreds of GB
  • RAID - this word describes the computer hard drive storage system, how much data can be stored in general. It typically starts at many terabytes.

As genomax states get as much RAM as possible hundreds of GB if possible.

ADD REPLYlink modified 19 months ago • written 19 months ago by Istvan Albert ♦♦ 79k
2
gravatar for genomax
19 months ago by
genomax64k
United States
genomax64k wrote:

Real RAM (when needed has no functional replacement). If you don't have enough of it you simply would not able to run certain jobs. So no matter what you choose make sure you get at least as much RAM as you will need for the largest jobs (+ 10% to account for future needs) and then plan for the rest of hardware.

ADD COMMENTlink written 19 months ago by genomax64k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 899 users visited in the last hour