Question: [cwl] referencing to list of paths in an array of files or direcories
0
gravatar for grange
13 months ago by
grange0
ASTRON, Netherlands Institute for Radio Astronomy
grange0 wrote:

I try to refer to a list of input paths in a CommandLineTool. In essence, my tool looks like this:

class: CommandLineTool
cwlVersion: v1.0

baseCommand:
  - python
inputs:
  - id: inpfiles
    type: 'Directory[]'

outputs: []
label: _do_magic
arguments:
  - position: 0
    prefix: '-c'
    valueFrom: |
      import magic
      magic.main(["$(inputs.inpfiles.path)"])

The main function of the magic script (not my script...) expects a dict of paths as input (I will probably have to play with the separator setting as well for this). The code I have now works for a single Directory object, but doesn't work. It does work when I refer to a single entry by using $inputs.inpfiles[0].path.

So my question is: how should I get a list of paths in an array of Directory objects?

cwl • 478 views
ADD COMMENTlink modified 13 months ago • written 13 months ago by grange0
2
gravatar for Michael R. Crusoe
13 months ago by
Common Workflow Language project
Michael R. Crusoe1.6k wrote:

Hello grange,

Since you're embedding your own code, you can get a JSON dump of the Directory object and walk over the "listing" attribute to extract the paths you need.

Try inpfiles=$(inputs.inpfiles) in the Python block

ADD COMMENTlink written 13 months ago by Michael R. Crusoe1.6k
0
gravatar for grange
13 months ago by
grange0
ASTRON, Netherlands Institute for Radio Astronomy
grange0 wrote:

Works like a charm! Thanks!

ADD COMMENTlink written 13 months ago by grange0
0
gravatar for grange
13 months ago by
grange0
ASTRON, Netherlands Institute for Radio Astronomy
grange0 wrote:

Actually, the issue now is that the variable that is passed contains a lot of data (including the list of files for each directory) which causes python to bail out with an OSError due to the argument list being too long. Any hints on keeping the data that is put on the command line short? I am using cwltool to run it, but I don't think that makes much of a difference.

ADD COMMENTlink modified 13 months ago • written 13 months ago by grange0

Answering my own question. Curious to see if this is indeed the neatest option:

class: CommandLineTool cwlVersion: v1.0

baseCommand:
  - python
inputs:
  - id: inpfiles
    inputBinding:
        position: 0
    type: 'Directory[]'

outputs: []
label: _do_magic
arguments:
  - position: 0
    prefix: '-c'
    valueFrom: |
      import sys
      import magic
      magic.main(sys.argv[1:])
ADD REPLYlink written 13 months ago by grange0

You can have it written to a file and then load json in the Python section

requirements:
  InitialWorkDirRequirement:
    listing:
    - entry: '{"_": $(inputs)}'
      entryname: inputs.json
ADD REPLYlink written 13 months ago by Michael R. Crusoe1.6k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1068 users visited in the last hour