Question: Memory allocation problems in 450k analysis
0
gravatar for arpit.singh1203
5.8 years ago by
India
arpit.singh120360 wrote:

I have 75 samples' data downloaded from TCGA. While running some of the steps in the pipelines like ChAMP, Minfi, etc., I get maximum memory allocation reached error. I have 4 GB RAM and i3 processor. Please suggest me possible alternatives or workarounds? Thanks much.

champ methylation minfi R • 1.5k views
ADD COMMENTlink modified 5.4 years ago by Shicheng Guo8.5k • written 5.8 years ago by arpit.singh120360
4
gravatar for Devon Ryan
5.8 years ago by
Devon Ryan98k
Freiburg, Germany
Devon Ryan98k wrote:

4 gigs of RAM isn't exactly a lot these days, particularly when you're talking about manipulating datasets of the size you're using. Try running this on a larger server.

ADD COMMENTlink written 5.8 years ago by Devon Ryan98k

Oh all right! I just thought maybe I could make-do by tweaking the code a little bit, which might increase the computational time. Is it possible? Could I break the process into smaller parts and solve this issue?

ADD REPLYlink written 5.8 years ago by arpit.singh120360
1

That I don't know, you'd need to look into exactly how the individual functions work. The biggest issue will be in any normalization step, where having the whole thing in memory is typically needed.

ADD REPLYlink written 5.8 years ago by Devon Ryan98k
2
gravatar for Shicheng Guo
5.4 years ago by
Shicheng Guo8.5k
Shicheng Guo8.5k wrote:

I definitely told you that it is the problem of memory. Methylation 450K array is very large data actually. It is very easy. The minimum memory should be > 450K*75 = 33.75G, Usually, I will do DMR analysis in memory=512G PCs. Such 512G memory can keep you do PCA, SVD, Cluster analysis, Classification and some other traditional data mining operation. If you have problems from memory, you can separate the data by the chromosomes and deal with them chromosome by chromosome.

ADD COMMENTlink modified 3.8 years ago • written 5.4 years ago by Shicheng Guo8.5k

What's the max sample size with 512G memory to run the analyses you mentioned?

ADD REPLYlink written 3.8 years ago by mbk0asis590
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1791 users visited in the last hour
_