We are running WGCNA on ~90,000 genes in a single block with 48 threads and 192GB of memory using the blockwiseModules
function.
WGCNA takes several dozen hours to compute the topological overlap matrix. We thought that 192GB would be sufficient for the analysis. But WGCNA chokes when exporting the TOM.
How can we estimate the memory required for blockwiseModules
to complete successfully? We have included the output below:
..Working on block 1 .
TOM calculation: adjacency..
..will use 48 parallel threads.
Fraction of slow calculations: 0.000000
..connectivity..
..matrix multiplication (system BLAS)..
..normalization..
..done.
..saving TOM for block 1 into file output/100000/wgcna/TOM-block.1.RData
....clustering..
Error in fastcluster::hclust(as.dist(dissTom), method = "average") :
Memory overflow.
Calls: blockwiseModules -> <Anonymous>
Execution halted