Snakemake + singularity fails to locate bwa index files.

Question

[jkim]

Hello,

I have no trouble with the bwa installed on my machine to align reads to the reference genome. However, when I use snakemake with singularity, it seems like singularity can't locate bwa index files. Could you tell me if there is an easy way to fix this issue?

rule bwa:
"""
Run bwa-mem
"""
input:
    fq=rules.pollux.output.fqgz,
output:
    outsam=temp("analysis/bwamem/{sample}.sam")
params:
    prefix="{sample}",
    idx=config['ref_modi']['index'],
log:
    stdout="logs/bwamem/{sample}.o",
    stderr="logs/bwamem/{sample}.e",
threads: 4
resources:
    mem_gb=20
container:
    "docker://biocontainers/bwa:v0.7.17_cv1"
shell:
    """
    bwa mem -M -t {threads}  {params.idx} {input.fq} > {output.outsam}
    """

This is the error message when I run snakemake using singularity.

(ABL1) (base) ➜  ABL1 git:(docker) ✗ snakemake -j1 --use-singularity -U bwa -p                                                                                               
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
bwa          1              1              1
total        1              1              1

Select jobs to execute...

[Tue Sep 13 09:49:37 2022]
rule bwa:
    input: analysis/pollux/ABL1.fq.gz
    output: analysis/bwamem/ABL1.sam
    log: logs/bwamem/ABL1.o, logs/bwamem/ABL1.e
    jobid: 5
    reason: Missing output files: analysis/bwamem/ABL1.sam
    wildcards: sample=ABL1
    resources: tmpdir=/tmp, mem_gb=20

        bwa mem -M -t 1  /media/hd1/jkim/genome/ABL1_NM005157.4/index/ucsc/hg19/bwa/NM_005157_exons_4_to_7 analysis/pollux/ABL1.fq.gz > analysis/bwamem/ABL1.sam

Activating singularity image /home/jaykim/jkim/singularity/bwa-v0.7.17.sif
INFO:    Converting SIF file to temporary sandbox...
WARNING: underlay of /usr/share/zoneinfo/Etc/UTC required more than 50 (56) bind mounts
[E::bwa_idx_load_from_disk] fail to locate the index files
INFO:    Cleaning up image...
[Tue Sep 13 09:49:41 2022]
Error in rule bwa:
    jobid: 5

I have no trouble running bwa without snakemake + singularity (I am using the bwa installed on my machine.)

(ABL1) (base) ➜  ABL1 git:(docker) ✗ bwa mem -M -t 1  /media/hd1/jkim/genome/ABL1_NM005157.4/index/ucsc/hg19/bwa/NM_005157_exons_4_to_7 analysis/pollux/ABL1.fq.gz > test.sam                
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 1599 sequences (444233 bp)...
[M::mem_process_seqs] Processed 1599 reads in 0.120 CPU sec, 0.120 real sec
[main] Version: 0.7.17-r1198-dirty
[main] CMD: bwa mem -M -t 1 /media/hd1/jkim/genome/ABL1_NM005157.4/index/ucsc/hg19/bwa/NM_005157_exons_4_to_7 analysis/pollux/ABL1.fq.gz
[main] Real time: 0.643 sec; CPU: 0.138 sec

[Jeremy Leipzig]

if the index is a file then it should be input, not a param

[jkim]

Thanks for the input. It is not a file but I think it's got something to do with Bind paths and mount. I can be totally wrong but when I tried --singularity-args "-B /path/outside/container/:/path/inside/container/" option, it worked - stackoverflow-post. I will look into it more.

[Jeremy Leipzig]

yeah bwa can't see /media if it's running inside a container, snakemake will create the outside:inside mapping relationships if knows those are crucial input. I would listing the index files as input and use basename to specify its parent directory in the command itself

[jkim]

I would listing the index files as input and use basename to specify its parent directory in the command itself.

Could you mind telling me how? I can do the first thing for sure (the index files as input) but I don't think I understand the second part.

[Jeremy Leipzig]

input: fq=rules.pollux.output.fqgz, amb="myref.amb", ann="myref.ann", bwt="myref.bwt", pac="myref.pac", sa="myref.sa" #use the paths on /media

shell: bwa mem -M -t {threads} `dirname {input.amb}` {input.fq} > {output.outsam}