cwltool SGE support for CWL parallel tasks
1
1
Entering edit mode
7.0 years ago

I am experimenting with CWL and cwltool. I have successfully written a workflow that can scatter jobs. However, cwltool runs them sequencially, whereas I would like to to use the Sun Grid Engine (SGE) to run jobs in parallel. I looked into Toil to run it but I have found it not capable of running the workflow so far. Here my questions:

  • Does cwltool support integration with SGE?
  • If not, is it planned, and if not, why? Would patches be accepted?
  • Beside Toil, are there working implementation that do interact with SGE?
  • What tools do you use to run cwl workflows with parrallel tasks?

Any other suggestion is welcome.

Thank you

cwl toil SGE • 2.9k views
ADD COMMENT
1
Entering edit mode
7.0 years ago
sayerskt ▴ 10

CWL tool I believe is always planned to just be local runner, and Toil is the only implementation currently with cluster support I believe. Can you post the workflow?

ADD COMMENT
0
Entering edit mode

I am getting this message: "CWL Directory inputs not yet supported in Toil"

ADD REPLY
1
Entering edit mode

That is correct, Toil's CWL support for v1.0 is ongoing: https://toil.readthedocs.io/en/releases-3.6.x/running/running.html#running-cwl-workflows

ADD REPLY

Login before adding your answer.

Traffic: 1458 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6