Question: cwltool SGE support for CWL parallel tasks
1
gravatar for Stefano Berri
14 months ago by
Stefano Berri4.0k
Cambridge, UK
Stefano Berri4.0k wrote:

I am experimenting with CWL and cwltool. I have successfully written a workflow that can scatter jobs. However, cwltool runs them sequencially, whereas I would like to to use the Sun Grid Engine (SGE) to run jobs in parallel. I looked into Toil to run it but I have found it not capable of running the workflow so far. Here my questions:

  • Does cwltool support integration with SGE?
  • If not, is it planned, and if not, why? Would patches be accepted?
  • Beside Toil, are there working implementation that do interact with SGE?
  • What tools do you use to run cwl workflows with parrallel tasks?

Any other suggestion is welcome.

Thank you

sge cwl toil • 683 views
ADD COMMENTlink modified 11 months ago by Biostar ♦♦ 20 • written 14 months ago by Stefano Berri4.0k
1
gravatar for sayerskt
14 months ago by
sayerskt10
sayerskt10 wrote:

CWL tool I believe is always planned to just be local runner, and Toil is the only implementation currently with cluster support I believe. Can you post the workflow?

ADD COMMENTlink written 14 months ago by sayerskt10

I am getting this message: "CWL Directory inputs not yet supported in Toil"

ADD REPLYlink written 14 months ago by Stefano Berri4.0k
1

That is correct, Toil's CWL support for v1.0 is ongoing: https://toil.readthedocs.io/en/releases-3.6.x/running/running.html#running-cwl-workflows

ADD REPLYlink written 13 months ago by Michael R. Crusoe970
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 714 users visited in the last hour