Question: cwltool SGE support for CWL parallel tasks
1
gravatar for Stefano Berri
3.6 years ago by
Stefano Berri4.1k
Cambridge, UK
Stefano Berri4.1k wrote:

I am experimenting with CWL and cwltool. I have successfully written a workflow that can scatter jobs. However, cwltool runs them sequencially, whereas I would like to to use the Sun Grid Engine (SGE) to run jobs in parallel. I looked into Toil to run it but I have found it not capable of running the workflow so far. Here my questions:

  • Does cwltool support integration with SGE?
  • If not, is it planned, and if not, why? Would patches be accepted?
  • Beside Toil, are there working implementation that do interact with SGE?
  • What tools do you use to run cwl workflows with parrallel tasks?

Any other suggestion is welcome.

Thank you

sge cwl toil • 1.8k views
ADD COMMENTlink modified 3.4 years ago by Biostar ♦♦ 20 • written 3.6 years ago by Stefano Berri4.1k
1
gravatar for sayerskt
3.6 years ago by
sayerskt10
sayerskt10 wrote:

CWL tool I believe is always planned to just be local runner, and Toil is the only implementation currently with cluster support I believe. Can you post the workflow?

ADD COMMENTlink written 3.6 years ago by sayerskt10

I am getting this message: "CWL Directory inputs not yet supported in Toil"

ADD REPLYlink written 3.6 years ago by Stefano Berri4.1k
1

That is correct, Toil's CWL support for v1.0 is ongoing: https://toil.readthedocs.io/en/releases-3.6.x/running/running.html#running-cwl-workflows

ADD REPLYlink written 3.6 years ago by Michael R. Crusoe1.8k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1330 users visited in the last hour