Question: CWL: print out command lines?
4
gravatar for sb10
2.9 years ago by
sb1040
sb1040 wrote:

Is there a tool that, given a cwl file and its necessary inputs and options, will simply print out the actual commands that need to be run for that workflow, without running them?

Better yet, one that also indicates the dependencies between the commands it outputs? So it would indicate in some way if the 3rd command it output was dependent on the first 2 commands it output completing first (but the first 2 are independent)?

cwl command • 1.4k views
ADD COMMENTlink modified 2.9 years ago by karl.sebby80 • written 2.9 years ago by sb1040

I would like to know this as well. I'm digging through the cwltool code now to try and make it work. It shouldn't be too hard (famous last words) since the command is printed out when you run a job. It is not an option when running cwltool though. Will let you know if I figure it out. Hoping somebody beats me to it though.

ADD REPLYlink modified 2.9 years ago • written 2.9 years ago by karl.sebby80

+1, would like to be able to summarise the full workflow without running samples. Thought perhaps the CWL viewer could be used as a basis for this?

ADD REPLYlink modified 2.9 years ago • written 2.9 years ago by bruce.moran830
1
gravatar for karl.sebby
2.9 years ago by
karl.sebby80
karl.sebby80 wrote:

OK. So I figured a few things out. If you are using the cwltool to run a job e.g. $ cwltool echo.cwl echo.yml where these files are

echo.cwl

cwlVersion: v1.0
class: CommandLineTool
stdout: echoOutput.txt
inputs:
  message:
    type: string
    inputBinding:
      position: 1
  message2:
    type: string
    inputBinding:
      position: 2
outputs:
  output: 
    type: stdout

echo.yml

message: "Hello World"
message2: "you doing good?"

Part of your output will look something like this:

[job echo.cwl] /tmp/tmpxxxx$ echo \
    'Hello world' \
    'you doing good?' > /tmp/tmpxxxx/echoOutput.txt
[job echo.cwl] completed success
Final process status is success

So the command is being generated and output to the screen. This happens using the logging module on lines 180-187 of cwltool/run.py. The commands are stored in a list called command_line which is a JobBase class attribute. To just output the command without running the job, I did the following:

1) In cwltool/main.py I added the argument 'commandline' to the argument parser after the --validate argument like so (above and below lines added for context):

exgroup.add_argument("--validate", action="store_true", help="Validate CWL document only.")
exgroup.add_argument("--commandline", action="store_true", help="Print the command line to command only.")
exgroup.add_argument("--print-supported-versions", action="store_true", help="Print supported CWL specs.")

2) In the main function of cwltool/main.py I set the default commandline value to False (above and below lines added for context)

'validate': False,
'commandline': False,
'enable_ga4gh_tool_registry': False,

3) then in cwltool/run.py at line 309 I added an if statement to print out the command line if the --commandline argument was set to true (above and below lines added for context again):

relink_initialworkdir(self.generatemapper, self.outdir, self.builder.outdir, inplace_update=self.inplace_update)
        if kwargs['commandline']:
            print(" ".join(self.command_line))
            exit(0)
        self._execute([], env, rm_tmpdir=rm_tmpdir, move_outputs=move_outputs)

Now if I run $ cwltool --commandline echo.cwl echo.yml I get

Resolved 'echo.cwl' to 'file:///.../echo.cwl'
echo Hello world you doing good?

Process finished with exit code 0

Hopefully this helps. Haven't tried it with anything more complex than my echo job.

ADD COMMENTlink modified 2.9 years ago • written 2.9 years ago by karl.sebby80

Thanks. Apparently it may also be possible by using custom executor and/or makeTool functions. I might give that a go at some point and report back here.

ADD REPLYlink written 2.8 years ago by sb1040

Hi sb10, Have you made any headway on this? I'm just circling back to this problem.

ADD REPLYlink written 2.7 years ago by karl.sebby80

Not yet had time to look at it. It's on the to-do list...

ADD REPLYlink written 2.7 years ago by sb1040
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1028 users visited in the last hour