Entering edit mode
7.5 years ago
kevin.o.oluoch
▴
70
Does the Common Workflow Language have if statements and for or while loops? If it does, what is the syntax?
Does the Common Workflow Language have if statements and for or while loops? If it does, what is the syntax?
It does not. This feature is under active discussion at https://groups.google.com/d/topic/common-workflow-language/JU7PSEKr97M/discussion
However, if you have several files that need to be processed (or an array that you want to iterate over for one or more other parameters) then you can use the scatter/gather feature http://www.commonwl.org/v1.0/Workflow.html#WorkflowStep
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
So just to confirm: There is currently no way of implementing a single workflow that, say, dynamically skips a step such as in the following example?
Good news: this is easily done if you know that there will always be some data that needs preprocessing and some that doesn't!
The "Pre-processing required?" step would need output the files in two segments: one to be pre-processed and one without. Javascript via and ExpressionTool can be used to avoid copying files around.
If sometime all data will either need or not-need pre-processing then it gets a little more complicated. In theory it might work if the various inputs and outputs were marked optional (a.k.a accepting
null
as a valid type) -- but I haven't personally tested that.The situation that CWL v1.0 doesn't at all support would be two or more divergent paths where the overall outputs would be radically different.
I'm sorry, but it seems that I didn't explain my question well. In fact, the whole data will indeed either need pre-processing or not need pre-processing (in which case the whole pre-processing
CommandLineTool
should ideally be entirely skipped).Moreover, "Pre-processing required?" wasn't actually supposed to be a
CommandLineTool
, but the logic that decides whether pre-processing should occur or not, based on the output of the "Inspect data" component (a string or file containing a string as suggested by Stefano). I am now realizing that the flowchart is unclear on this because I have used the same arrows both for decisions and data flow. What I meant is that the data should actually flow from either "Data" or "Inspect data" (which one is irrelevant in this example) to either "Pre-process data" or "Process data" depending on the decision, but not to both.Hope it is clearer now.
I am a beginner with cwl, but here my approach to conditional events. Make the workflow linear, like this:
Inspect data | Preprocess-Required? | Pre-process data | Process Data
L'et's say that Preprocess-Required spit out a file with "YES" or "NO" Then preprocess data read that file and spit out either the same input files (if previous step said "NO") or the preprocessed files. That, of course, requires that you have some control over the Pre-process data step and can add the ability to take this decision. Basically, your "if logic" need to get into the step.
Thanks a lot for the suggestion.
However, I would like to avoid wrapping scripts/tools or even refactoring them (assuming I have control over the code) just to meet such a specific purpose. It would be kinda weird to end up having a flag
--execute
or--pass-through
in all future scripts, no? :-) Also, it would pretty much defeat the purpose of having a modular workflow framework in my opinion. By the way, I would feel similarly uneasy about tampering withCommandLineTools
in a way suggested by Michael, i.e. setting all input/output ports optional, if they're actually not meant to be optional. A global, CWL-supported--pass-through
option on the level ofWorkflows
would be something different of course.Anyway, my current workaround is to split up the workflow and outsource the logic to a global wrapper script (or web service or whatever) that takes care of coordinating sub-workflows. At least for me that seems to be the least invasive option.
Thanks, I hope the feature will be up soon. I'll be following the discussion
I think this has been implemented ( https://www.commonwl.org/v1.2/Workflow.html#Conditional_execution_(Optional) ) but I am having trouble finding solid official docs on examples of implementing all the features of it.