Entering edit mode
11.2 years ago
Paul ▴ 760
I've got a massive dataset of 160 Affy Exon arrays that I need to quantiles normalize at probe level (the reasons for which are complicated!), which means ~6 million probes per sample.
Trying to load the into memory maxes out 32gigs of memory on one of our cluster nodes.
I clearly need to find a way to do this without loading it into memory, would anyone perhaps have a pointer/suggestion?