Question: cwltool SGE support for CWL parallel tasks
gravatar for Stefano Berri
24 months ago by
Stefano Berri4.1k
Cambridge, UK
Stefano Berri4.1k wrote:

I am experimenting with CWL and cwltool. I have successfully written a workflow that can scatter jobs. However, cwltool runs them sequencially, whereas I would like to to use the Sun Grid Engine (SGE) to run jobs in parallel. I looked into Toil to run it but I have found it not capable of running the workflow so far. Here my questions:

  • Does cwltool support integration with SGE?
  • If not, is it planned, and if not, why? Would patches be accepted?
  • Beside Toil, are there working implementation that do interact with SGE?
  • What tools do you use to run cwl workflows with parrallel tasks?

Any other suggestion is welcome.

Thank you

sge cwl toil • 1.1k views
ADD COMMENTlink modified 21 months ago by Biostar ♦♦ 20 • written 24 months ago by Stefano Berri4.1k
gravatar for sayerskt
24 months ago by
sayerskt10 wrote:

CWL tool I believe is always planned to just be local runner, and Toil is the only implementation currently with cluster support I believe. Can you post the workflow?

ADD COMMENTlink written 24 months ago by sayerskt10

I am getting this message: "CWL Directory inputs not yet supported in Toil"

ADD REPLYlink written 24 months ago by Stefano Berri4.1k

That is correct, Toil's CWL support for v1.0 is ongoing:

ADD REPLYlink written 23 months ago by Michael R. Crusoe1.4k
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1799 users visited in the last hour