I am new into CWL coding, I am trying to implement a workflow and I ran across some issues . I am not able to mention the output file name outputted by one step as input for another step in my YML script . It says that such file is not present in the folder already. That file is being generated on "output7". I tried with staging the input file also , but that also throws error.I am attaching both CWL and YML file along with this post.
CWL:
cwlVersion: cwl:draft-3
class: Workflow
inputs:
- id: output7
type: string
- id: reference7
type: File
- id: output8
type: string
- id: reference9
type: File
- id: numThreads9
type: int
- id: input9
type: File
- id: output9
type: string
outputs:
- id: workflowOutput
type: File
source: "#picard-ReorderSam/result"
- id: workflowOutput1
type: File
source: "#samtoolsIndex/result"
- id: workflowOutput2
type: File
source: "#gatk-RealignerTargetCreator/result"
steps:
- id: picard-ReorderSam
run: picard-ReorderSam.cwl
inputs:
- id: input
source: "#picard-AddorRep/result"
- id: output
source: "#output7"
- id: reference
source: "#reference7"
outputs:
- id: result
- id: samtoolsIndex
run: samtools-index.cwl
inputs:
- id: input
source: "#picard-ReorderSam/result"
- id: output
source: "#output8"
outputs:
- id: result
- id: gatk-RealignerTargetCreator
run: gatk-RealignerTargetCreator.cwl
inputs:
- id: reference
source: "#reference9"
- id: numThreads
source: "#numThreads9"
- id: input
source: "#input9"
- id: output
source: "#output9"
outputs:
- id: result
YAML script :
output7: excapgene1normoutstp8.bam
reference7:
class: File
path: /home/cc/ReferenceData/genome_sorted.fa
output8: excapgene1normoutstp8.bai
reference9:
class: File
path: /home/cc/ReferenceData/genome_sorted.fa
numThreads9: 10
input9:
class: File
path: excapgene1normoutstp8.bam
output9: realignment_targets.list
**Error message generated **:
ValidationException: exomecap-part1-job.yml:43:3: [Errno 2] No such file or directory: '/home/cc/Test_CWL/excapgene1normoutstp8.bam'
[step gatk-RealignerTargetCreator] Cannot make job: exomecap-part1-job.yml:43:3: [Errno 2] No such file or directory: '/home/cc/Test_CWL/excapgene1normoutstp8.bam'
[workflow exomecap-part1.cwl] outdir is /tmp/tmpE4Tn6Q
{}
Final process status is permanentFail
I tried to stage the file without outputting the file into the file and inputting for each step , am not able to stage the output from output7 to input9.
CWL :
cwlVersion: cwl:draft-3
class: Workflow
inputs:
- id: output7
type: string
- id: reference7
type: File
- id: output8
type: string
- id: reference9
type: File
- id: numThreads9
type: int
- id: output9
type: string
outputs:
- id: workflowOutput
type: File
source: "#gatk-RealignerTargetCreator/result"
steps:
- id: picard-ReorderSam
run: picard-ReorderSam.cwl
inputs:
- id: input
source: "#picard-AddorRep/result"
- id: output
source: "#output7"
- id: reference
source: "#reference7"
outputs:
- id: result
- id: samtoolsIndex
run: samtools-index.cwl
inputs:
- id: input
source: "#picard-ReorderSam/result"
- id: output
source: "#output8"
outputs:
- id: result
- id: gatk-RealignerTargetCreator
run: gatk-RealignerTargetCreator.cwl
inputs:
- id: reference
source: "#reference9"
- id: numThreads
source: "#numThreads9"
- id: input
source: "#picard-ReorderSam/result"
- id: output
source: "#output9"
outputs:
- id: result
YML script :
output7: excapgene1normoutstp8.bam
reference7:
class: File
path: /home/cc/ReferenceData/genome_sorted.fa
output8: excapgene1normoutstp8.bai
reference9:
class: File
path: /home/cc/ReferenceData/genome_sorted.fa
numThreads9: 10
output9: realignment_targets.list
Error generated:
ValidationException: [Errno 2] No such file or directory: '/tmp/tmp06gkDT/excapgene1normoutstp8.bai'
[step gatk-RealignerTargetCreator] Cannot make job: [Errno 2] No such file or directory: '/tmp/tmp06gkDT/excapgene1normoutstp8.bai'
[workflow exomecap-part1.cwl] outdir is /tmp/tmpuJgV1v
{}
Final process status is permanentFail
I tried to stage the input files from previous step output but for processing the "gatk-RealignerTargetCreator" it requires ".Bai" file outputted by SamtoolIndex as secondary file along with the input file "excapgene1normoutstp8.bam " which is outputed by "picard-ReorderSam".
Please kindly help me to solve this issue.
Hello akashmdr90,
The first workflow listed is missing the
picard-AddorRep
step. Also, can you link to a GitHub repo with these files? It is hard to debug without having access to the underlying too definitions.Also, may I ask why you are using the
draft-3
CWL syntax? I don't know of any system that supportsdraft-3
version of CWL and doesn't also support thev1.0
version that came after it. You can use thecwl-upgrader
to upgrade yourdraft-3
files quickly tov1.0
: https://pypi.python.org/pypi/cwl-upgrader