EIGENSOFT - smartpca

Question

[camlec]

Hello,

I'm new to this forum so forgive me if this post isn't the way it should be.

Im trying to run smartpca but am getting the following error:

smartpca -p smartpca_2.par
parameter file: smartpca_2.par

\### THE INPUT PARAMETERS

\##PARAMETER NAME: VALUE

genotypename: example.eigenstratgeno

snpname: example.snp

indivname: example.ind

evecoutname: eigenstrat_outlier_removed.evec

evaloutname: eigenstrat_outlier_removed.eval

outlieroutname: eigenstrat_outlier_removed.txt

numoutlieriter: 5

numoutlierevec: 10

outliersigmathresh: 6

usenorm: YES

familynames: NO

\## smartpca version: 13050

norm used

number of samples used: 88733 number of snps used: 4057

fatalx:

Unable to allocate 7873545289 unit(s) for item

Aborted (core dumped)

Could anyone tell me why this is happening ? I tried using the ped-map files and also tried after converting ped-map to eigenstrat format but the error remains the same. I even tried running it on a compute cluster.

Thanks a lot.

[h.mon]

Please use the formatting bar (especially the code option) to present your post better.

[camlec]

It's funny because max_virtual_mem on Sun Grid Engine shows 1G for ~ 90k samples and breaks while for 5k,10k the max_virtual_mem is 1.5, 3G and it runs to completion. When I raise to ~17k samples, it crashes again.

I'm trying flashpca2 instead since I dont know how to get smartpca to work on this large cohort size.

And it's a bus error (exit 137) for 17k and 20k while it's an allocation error for ~90k samples which also seems weird.

[h.mon]

Please use the ADD COMMENT button to address Jean-Karim Heriche directly, instead of adding an answer. Could you correct this issue?

Answer 1

[Jean-Karim Heriche]

This is a memory allocation error. Either you don't have enough RAM or there's something wrong with your data. The unit of memory allocation depends on the type of data being treated. Assuming it was processing numbers represented as floating point of double precision, a unit would be 8 bytes so 7873545289 units would be about 59 GB of memory.

[camlec]

It's funny because max_virtual_mem on Sun Grid Engine shows 1G for ~ 90k samples and breaks while for 5k,10k the max_virtual_mem is 1.5, 3G and it runs to completion. When I raise to ~17k samples, it crashes again.

I'm trying flashpca2 instead since I dont know how to get smartpca to work on this large cohort size.

And it's a bus error (exit 137) for 17k and 20k while it's an allocation error for ~90k samples which also seems weird.

[Jean-Karim Heriche]

So you're running this on a cluster. You need to make sure that you specify your job requirement adequately. I am not familiar with SGE but depending on your cluster configuration it is possible that if you don't reserve memory for your job, another one could run on the same nodes and take all the memory.