Supergnova stuck in tutorial

Question

[Gerard]

I am trying to use Supergnova (https://github.com/YiliangTracyZhang/SUPERGNOVA) to calculate local genetic correlations from multiple GWAS summary statistics. To start with, I'm trying to run the tutorial they provide, but the final command:

python3 supergnova.py ./data/sumstats/ASD.txt.sumstats.gz ./data/sumstats/CP.txt.sumstats.gz \
--N1 46351 \
--N2 257828 \
--bfile data/bfiles/eur_chr@_SNPmaf5 \
--partition data/partition/eur_chr@.bed \
--out results.txt

gets stuck without showing any error after it has been running for more than 24 hours. From what I have read, the expected process time is between 30 min and 1 hour, which clearly suggests that something is not working properly.

I have created a conda environment with the packages required to use Supergnova (Python 3, numpy scipy, pandas, sklearn, bitarray) but since there are no previous steps and I am simply following all the steps in the tutorial, I don't know what might be going on. I would appreciate if someone could suggest me something I could try to do to solve it.

Thanks.

[GenoMax]

Without an error it would be difficult to diagnose this. Kill the process if it has not produced any output for 24 h. You may want to specify a value for threads --threads 2 and see if that helps.

[Gerard]

Thanks for your answer and suggestion.

I am copying the message that the terminal returned after I manually stopped it if it can be helpful:

File "/home/gerard/SUPERGNOVA/supergnova.py", line 93, in <module>
    pipeline(parser.parse_args())
  File "/home/gerard/SUPERGNOVA/supergnova.py", line 65, in pipeline
    out = calculate(args.bfile, bed, args.thread, gwas_snps, ld_scores, N1, N2, pheno_corr, pheno_corr_var)
  File "/home/gerard/SUPERGNOVA/calculate.py", line 193, in calculate
    all_dfs.append(_supergnova(cur_bfile, partition, thread, gwas_snps, ld_scores, n1, n2, pheno_corr, pheno_corr_var))
  File "/home/gerard/SUPERGNOVA/calculate.py", line 170, in _supergnova
    pool.join()
  File "/home/gerard/anaconda3/envs/superg/lib/python3.9/multiprocessing/pool.py", line 662, in join
    self._worker_handler.join()
  File "/home/gerard/anaconda3/envs/superg/lib/python3.9/threading.py", line 1060, in join
    self._wait_for_tstate_lock()
  File "/home/gerard/anaconda3/envs/superg/lib/python3.9/threading.py", line 1080, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
KeyboardInterrupt

I forgot to mention that I have already tried to use a different number of threads but the problem persisted.

[GenoMax]

Which version of python3 are you using on your system? It appears based on the message above that the job was possibly stuck on threading step. Can you try to run using a single thread --threads 1 and see if that allows the job to proceed.

[Gerard]

I am using Python 3.9.13.

I also thought that the error was coming from the use of several threads, but I tried to run it using only one and the job didn't proceed either.

[GenoMax]

You can try creating an issue (with this error) https://github.com/YiliangTracyZhang/SUPERGNOVA/issues to see if the developer would be able to help.

Answer 1

[Gerard]

Thanks for all the suggestions. I created an issue but, unfortunately, was not answered.

I managed to solve the issues I was having changing the versions of the Python packages SUPERGNOVA requires. I will let them noted here in case anyone needs them:

Numpy 1.19.5 Scipy 1.5.4 pandas 0.25.3 bitarray 2.1.0 sklearn = scikit-learn 0.24.2

[Murat]

Hi Gerard, I am also using the tutorial and found your answer. The ftp links are not working any longer; is there any other source to download bfiles and partitions?

Thanks, Murat

[GenoMax]

Your best option may be to email the author (and/or the senior author from the paper). Let them know that the FTP links mentioned on GitHub are no longer working and you need the files mentioned in them.

Post alternate source information (if you receive it from authors) here.

[Murat]

Hi GenoMax, Thank you for your advice. I sent an email to them and have already opened an issue on Github. Let see if there is any other source to get this data.

Thanks!