fail to open file - bwa mem using CWL

Question

[skanwal]

Hi,

I am trying to run simple two step CWL workflow, using v1.0. Step 1: Create sequence dictionary for reference Step 2: Align input fastq reads with reference

Step1 runs fine but step 2 (bwa mem) throws the following error:

 [job bwa-mem] /Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/outputFiles/tmpOJa_OJ$ docker \
    run \
    -i \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/inputFiles/hg19.fasta.ann:/private/var/lib/cwl/stg744d0af7-90e5-48c2-9039-998a29de796f/hg19.fasta.ann:ro' \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/inputFiles/hg19.dict:/private/var/lib/cwl/stg744d0af7-90e5-48c2-9039-998a29de796f/hg19.dict:ro' \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/inputFiles/hg19.fasta.sa:/private/var/lib/cwl/stg744d0af7-90e5-48c2-9039-998a29de796f/hg19.fasta.sa:ro' \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/outputFiles/tmpDk44EY/hg19.dict:/private/var/lib/cwl/stg9ffab43b-2c15-4fe3-8011-016f50ea234f/hg19.dict:ro' \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/inputFiles/hg19.fasta.amb:/private/var/lib/cwl/stg744d0af7-90e5-48c2-9039-998a29de796f/hg19.fasta.amb:ro' \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/inputFiles/hg19.fasta:/private/var/lib/cwl/stg744d0af7-90e5-48c2-9039-998a29de796f/hg19.fasta:ro' \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/inputFiles/hg19.fasta.fai:/private/var/lib/cwl/stg744d0af7-90e5-48c2-9039-998a29de796f/hg19.fasta.fai:ro' \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/inputFiles/hg19.fasta.pac:/private/var/lib/cwl/stg744d0af7-90e5-48c2-9039-998a29de796f/hg19.fasta.pac:ro' \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/inputFiles/hg19.fasta.bwt:/private/var/lib/cwl/stg744d0af7-90e5-48c2-9039-998a29de796f/hg19.fasta.bwt:ro' \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/outputFiles/tmpOJa_OJ:/private/var/spool/cwl:rw' \
    '--volume=/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/outputFiles/tmpKGeIbn:/tmp:rw' \
    --workdir=/private/var/spool/cwl \
    --read-only=true \
    --log-driver=none \
    --user=1000 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/private/var/spool/cwl \
    --env=PATH=/usr/local/bin/:/usr/bin:/bin \
    scidap/bwa:v0.7.12 \
    bwa \
    mem \
    /private/var/lib/cwl/stg744d0af7-90e5-48c2-9039-998a29de796f/hg19.fasta \
    '/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/Fastq-files/FQ_1_small.fastq' \
    '/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/Fastq-files/FQ_2_small.fastq' > /Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/outputFiles/tmpOJa_OJ/bwa-mem.sam
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[E::main_mem] **fail to open file `/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/Fastq-files/FQ_1_small.fastq'.**
[job bwa-mem] completed permanentFail
[step bwa-mem] completed permanentFail
[workflow GATK-workflow-v1.0.cwl] outdir is /Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/outputFiles/tmpUpe8_W
{
    "bwamem_output": {
        "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709", 
        "basename": "bwa-mem.sam", 
        "location": "file:///Users/sehrish/Google%20Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/bwa-mem.sam", 
        "path": "/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/bwa-mem.sam", 
        "class": "File", 
        "size": 0
    }, 
    "ReferenceSequenceDictionary": {
        "checksum": "sha1$121da34bdcd3f3febd2700298cefc7c3e9e8e2f6", 
        "basename": "hg19.dict", 
        "location": "file:///Users/sehrish/Google%20Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/hg19.dict", 
        "path": "/Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/hg19.dict", 
        "class": "File", 
        "size": 13712
    }
}
Final process status is permanentFail

However, I have tried running bwa mem locally (i.e. the tool itself with out cwl) and it runs fine. Wonder if it is some path issue with docker ? or cwl?

Any help will be much appreciated. Thanks.

Answer 1

[solomon.shorser]

It looks to me like the path /Users/sehrish/Google Drive/AnalysisGalaxyCpipeCWL/GATK-worflow/v1.0/Fastq-files/FQ_1_small.fastq is not accessible from within the docker container. You might want to specify that FQ_1_small.fastq is an input file so that it will be mounted into the container.

[skanwal]

Thanks for the reply. I have solved this issue by making some changes in the yaml file for workflow. I was not specifying the input path correctly in the job file and had some issues in specifying the array of files as input to bwa-mem. It's finally resolved :)