Snakemake + singularity fails to locate bwa index files.
0
1
Entering edit mode
21 months ago
jkim ▴ 170

Hello,

I have no trouble with the bwa installed on my machine to align reads to the reference genome. However, when I use snakemake with singularity, it seems like singularity can't locate bwa index files. Could you tell me if there is an easy way to fix this issue?

rule bwa:
"""
Run bwa-mem
"""
input:
    fq=rules.pollux.output.fqgz,
output:
    outsam=temp("analysis/bwamem/{sample}.sam")
params:
    prefix="{sample}",
    idx=config['ref_modi']['index'],
log:
    stdout="logs/bwamem/{sample}.o",
    stderr="logs/bwamem/{sample}.e",
threads: 4
resources:
    mem_gb=20
container:
    "docker://biocontainers/bwa:v0.7.17_cv1"
shell:
    """
    bwa mem -M -t {threads}  {params.idx} {input.fq} > {output.outsam}
    """

This is the error message when I run snakemake using singularity.

(ABL1) (base) ➜  ABL1 git:(docker) ✗ snakemake -j1 --use-singularity -U bwa -p                                                                                               
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
bwa          1              1              1
total        1              1              1

Select jobs to execute...

[Tue Sep 13 09:49:37 2022]
rule bwa:
    input: analysis/pollux/ABL1.fq.gz
    output: analysis/bwamem/ABL1.sam
    log: logs/bwamem/ABL1.o, logs/bwamem/ABL1.e
    jobid: 5
    reason: Missing output files: analysis/bwamem/ABL1.sam
    wildcards: sample=ABL1
    resources: tmpdir=/tmp, mem_gb=20

        bwa mem -M -t 1  /media/hd1/jkim/genome/ABL1_NM005157.4/index/ucsc/hg19/bwa/NM_005157_exons_4_to_7 analysis/pollux/ABL1.fq.gz > analysis/bwamem/ABL1.sam

Activating singularity image /home/jaykim/jkim/singularity/bwa-v0.7.17.sif
INFO:    Converting SIF file to temporary sandbox...
WARNING: underlay of /usr/share/zoneinfo/Etc/UTC required more than 50 (56) bind mounts
[E::bwa_idx_load_from_disk] fail to locate the index files
INFO:    Cleaning up image...
[Tue Sep 13 09:49:41 2022]
Error in rule bwa:
    jobid: 5

I have no trouble running bwa without snakemake + singularity (I am using the bwa installed on my machine.)

(ABL1) (base) ➜  ABL1 git:(docker) ✗ bwa mem -M -t 1  /media/hd1/jkim/genome/ABL1_NM005157.4/index/ucsc/hg19/bwa/NM_005157_exons_4_to_7 analysis/pollux/ABL1.fq.gz > test.sam                
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 1599 sequences (444233 bp)...
[M::mem_process_seqs] Processed 1599 reads in 0.120 CPU sec, 0.120 real sec
[main] Version: 0.7.17-r1198-dirty
[main] CMD: bwa mem -M -t 1 /media/hd1/jkim/genome/ABL1_NM005157.4/index/ucsc/hg19/bwa/NM_005157_exons_4_to_7 analysis/pollux/ABL1.fq.gz
[main] Real time: 0.643 sec; CPU: 0.138 sec
singularity snakemake bwa • 1.4k views
ADD COMMENT
0
Entering edit mode

if the index is a file then it should be input, not a param

ADD REPLY
0
Entering edit mode

Thanks for the input. It is not a file but I think it's got something to do with Bind paths and mount. I can be totally wrong but when I tried --singularity-args "-B /path/outside/container/:/path/inside/container/" option, it worked - stackoverflow-post. I will look into it more.

ADD REPLY
1
Entering edit mode

yeah bwa can't see /media if it's running inside a container, snakemake will create the outside:inside mapping relationships if knows those are crucial input. I would listing the index files as input and use basename to specify its parent directory in the command itself

ADD REPLY
0
Entering edit mode

I would listing the index files as input and use basename to specify its parent directory in the command itself.

Could you mind telling me how? I can do the first thing for sure (the index files as input) but I don't think I understand the second part.

ADD REPLY
1
Entering edit mode

input: fq=rules.pollux.output.fqgz, amb="myref.amb", ann="myref.ann", bwt="myref.bwt", pac="myref.pac", sa="myref.sa" #use the paths on /media

shell: bwa mem -M -t {threads} `dirname {input.amb}` {input.fq} > {output.outsam}

ADD REPLY

Login before adding your answer.

Traffic: 884 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6