Question: CWL: Loops and if conditionals
3
gravatar for kevin.o.oluoch
13 months ago by
Kenya
kevin.o.oluoch60 wrote:

Does the Common Workflow Language have if statements and for or while loops? If it does, what is the syntax?

cwl loops • 808 views
ADD COMMENTlink modified 13 months ago by Michael R. Crusoe970 • written 13 months ago by kevin.o.oluoch60
2
gravatar for Michael R. Crusoe
13 months ago by
currently based out of Vilnius, Lithuania / Common Workflow Language project
Michael R. Crusoe970 wrote:

It does not. This feature is under active discussion at https://groups.google.com/d/topic/common-workflow-language/JU7PSEKr97M/discussion

However, if you have several files that need to be processed (or an array that you want to iterate over for one or more other parameters) then you can use the scatter/gather feature http://www.commonwl.org/v1.0/Workflow.html#WorkflowStep

ADD COMMENTlink written 13 months ago by Michael R. Crusoe970
1

So just to confirm: There is currently no way of implementing a single workflow that, say, dynamically skips a step such as in the following example?

Skipping a step during CWL execution

ADD REPLYlink modified 13 months ago • written 13 months ago by unique_g80

Good news: this is easily done if you know that there will always be some data that needs preprocessing and some that doesn't!

The "Pre-processing required?" step would need output the files in two segments: one to be pre-processed and one without. Javascript via and ExpressionTool can be used to avoid copying files around.

If sometime all data will either need or not-need pre-processing then it gets a little more complicated. In theory it might work if the various inputs and outputs were marked optional (a.k.a accepting null as a valid type) -- but I haven't personally tested that.

The situation that CWL v1.0 doesn't at all support would be two or more divergent paths where the overall outputs would be radically different.

ADD REPLYlink written 13 months ago by Michael R. Crusoe970

I'm sorry, but it seems that I didn't explain my question well. In fact, the whole data will indeed either need pre-processing or not need pre-processing (in which case the whole pre-processing CommandLineTool should ideally be entirely skipped).

Moreover, "Pre-processing required?" wasn't actually supposed to be a CommandLineTool, but the logic that decides whether pre-processing should occur or not, based on the output of the "Inspect data" component (a string or file containing a string as suggested by Stefano). I am now realizing that the flowchart is unclear on this because I have used the same arrows both for decisions and data flow. What I meant is that the data should actually flow from either "Data" or "Inspect data" (which one is irrelevant in this example) to either "Pre-process data" or "Process data" depending on the decision, but not to both.

Hope it is clearer now.

ADD REPLYlink written 13 months ago by unique_g80

I am a beginner with cwl, but here my approach to conditional events. Make the workflow linear, like this:

Inspect data | Preprocess-Required? | Pre-process data | Process Data

L'et's say that Preprocess-Required spit out a file with "YES" or "NO" Then preprocess data read that file and spit out either the same input files (if previous step said "NO") or the preprocessed files. That, of course, requires that you have some control over the Pre-process data step and can add the ability to take this decision. Basically, your "if logic" need to get into the step.

ADD REPLYlink written 13 months ago by Stefano Berri4.0k

Thanks a lot for the suggestion.

However, I would like to avoid wrapping scripts/tools or even refactoring them (assuming I have control over the code) just to meet such a specific purpose. It would be kinda weird to end up having a flag --execute or --pass-through in all future scripts, no? :-) Also, it would pretty much defeat the purpose of having a modular workflow framework in my opinion. By the way, I would feel similarly uneasy about tampering with CommandLineTools in a way suggested by Michael, i.e. setting all input/output ports optional, if they're actually not meant to be optional. A global, CWL-supported --pass-through option on the level of Workflows would be something different of course.

Anyway, my current workaround is to split up the workflow and outsource the logic to a global wrapper script (or web service or whatever) that takes care of coordinating sub-workflows. At least for me that seems to be the least invasive option.

ADD REPLYlink modified 13 months ago • written 13 months ago by unique_g80

Thanks, I hope the feature will be up soon. I'll be following the discussion

ADD REPLYlink written 13 months ago by kevin.o.oluoch60
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1343 users visited in the last hour