GATK CNV pipeline gcnvkernel import error
0
0
Entering edit mode
4 months ago
Maverick ▴ 10

Hello, I am trying to run GATK's CNV pipeline.

Has anyone faced a similar error like this? I am able to run the entire pipeline at my login node from the same python environment. But when I submit my job script to the compute node of our HPC, the import gcnvkernel throws this error.

22:50:32.029 INFO  DetermineGermlineContigPloidy - Initializing engine
22:50:32.033 DEBUG ScriptExecutor - Executing:
22:50:32.033 DEBUG ScriptExecutor -   python
22:50:32.033 DEBUG ScriptExecutor -   -c
22:50:32.033 DEBUG ScriptExecutor -   import gcnvkernel

ERROR (theano.gof.opt): Optimization failure due to: local_subtensor_merge
ERROR (theano.gof.opt): node: Subtensor{:int64:}(log_emission_tc[0:], ScalarFromTensor.0)
ERROR (theano.gof.opt): TRACEBACK:
ERROR (theano.gof.opt): Traceback (most recent call last):
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/opt.py", line 2034, in process_node
    replacements = lopt.transform(node)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/tensor/opt.py", line 3091, in local_subtensor_merge
    ushape[pos_2]))
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/tensor/opt.py", line 3039, in merge_two_slices
    start = pre_greedy_local_optimizer(list_opt, start)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/opt.py", line 2923, in pre_greedy_local_optimizer
    list_optimizations, out, {}, 0)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/opt.py", line 2895, in local_recursive_function
    depth + 1)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/opt.py", line 2895, in local_recursive_function
    depth + 1)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/opt.py", line 2907, in local_recursive_function
    ret = opt.transform(node)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/tensor/opt.py", line 6516, in constant_folding
    no_recycling=[], impl=impl)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/op.py", line 955, in make_thunk
    no_recycling)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/op.py", line 858, in make_c_thunk
    output_storage=node_output_storage)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/cc.py", line 1217, in make_thunk
    keep_lock=keep_lock)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/cc.py", line 1157, in __compile__
    keep_lock=keep_lock)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/cc.py", line 1624, in cthunk_factory
    key=key, lnk=self, keep_lock=keep_lock)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/cmodule.py", line 1189, in module_from_key
    module = lnk.compile_cmodule(location)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/cc.py", line 1527, in compile_cmodule
    preargs=preargs)
  File "~/miniconda3/envs/gatk/lib/python3.6/site-packages/theano/gof/cmodule.py", line 2411, in compile_str
    (status, compile_stderr.replace('\n', '. ')))
Exception: ('Compilation failed (return status=1): In file included from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/syslimits.h:7:0,.       from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/limits.h:34,.      from ~/miniconda3/envs/gatk/include/python3.6m/Python.h:11,.  from /projectnb/pegasus/~/.theano/compiledir_Linux-3.10-el7.x86_64-x86_64-with-centos-7.6.1810-Core-x86_64-3.6.10-64/tmpi51kx06_/mod.cpp:1:. /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/limits.h:168:61: error: no include path in which to search for limits.h.  #include_next <limits.h>  /* recurse down to the real one */.                        ^. In file included from /projectnb/pegasus/~/.theano/compiledir_Linux-3.10-el7.x86_64-x86_64-with-centos-7.6.1810-Core-x86_64-3.6.10-64/tmpi51kx06_/mod.cpp:1:0:. ~/miniconda3/envs/gatk/include/python3.6m/Python.h:25:19: fatal error: stdio.h: No such file or directory.  #include <stdio.h>.  ^. compilation terminated.. ', '[InplaceDimShuffle{}(TensorConstant{0})]')

I activate the environment and load all required modules as mentioned below in my job script:

source ~/miniconda3/etc/profile.d/conda.sh

module load java/17
conda activate gatk

Are there any changes that need to be made when submitting a job to HPC? I can see that the error says certain files like stdio.h and limits.h are not there but they are present as seen below -

find $CONDA_PREFIX -name limits.h
~/miniconda3/envs/gatk/x86_64-conda-linux-gnu/include/c++/7.5.0/tr1/limits.h
~/miniconda3/envs/gatk/x86_64-conda-linux-gnu/sysroot/usr/include/linux/limits.h
~/miniconda3/envs/gatk/x86_64-conda-linux-gnu/sysroot/usr/include/limits.h
~/miniconda3/envs/gatk/lib/gcc/x86_64-conda-linux-gnu/7.5.0/include-fixed/limits.h
~/miniconda3/envs/gatk/lib/gcc/x86_64-conda-linux-gnu/7.5.0/install-tools/include/limits.h

 find $CONDA_PREFIX -name stdio.h
~/miniconda3/envs/gatk/x86_64-conda-linux-gnu/include/c++/7.5.0/tr1/stdio.h
~/miniconda3/envs/gatk/x86_64-conda-linux-gnu/sysroot/usr/include/bits/stdio.h
~/miniconda3/envs/gatk/x86_64-conda-linux-gnu/sysroot/usr/include/stdio.h

Thank you for your time.

Theano gcnvkernel GATK CNV HPC • 350 views
ADD COMMENT
0
Entering edit mode

Incase anyone has a similar issue - the HPC's gcc was overriding the gcc provided by the gatk package in the conda environment.

I followed conda-gcc-does-not-replace-system-gcc to create a symbolic link and it was sorted.

It was however followed by another error "missing crypt.h". Depending on the conda version the file can be placed in a location other than the anticipated one. I followed missing crypt.h to copy and paste the file and the script now runs without issues.

ADD REPLY

Login before adding your answer.

Traffic: 852 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6