Question: Missing CWL files in a Directory when passed as input
1
gravatar for asha
10 months ago by
asha20
asha20 wrote:

I wasn't sure if this was a question or a bug, so decided to first post a question about it. I can file an issue if it's confirmed to be a bug.

It looks like files are not mounted properly in directories when they're passed from one tool to another. Note that the files are tracked properly by CWL; they're just not mapped to the directory.

An example workflow is shown below. The output of ls is just an empty directory. The expected output is a full list of files and their subdirectories. I ran it using cwl-runner v1.0 and cwltool v1.0.20190831161204. Is there something that I'm missing when operating on directories in CWL?

class: Workflow
cwlVersion: v1.0
requirements:
  InlineJavascriptRequirement: {}
  SubworkflowFeatureRequirement: {}

inputs: {}
steps:
  gen_dir:
    run:
      class: CommandLineTool
      baseCommand: [bash, -c, 'mkdir -p test/subdir; echo hello > test/hello.txt; echo world > test/subdir/world.txt']
      inputs: {}
      outputs:
        dir:
          type: Directory
          outputBinding:
            glob: 'test'
    in: {}
    out: [dir]

  ls:
    run:
      class: CommandLineTool
      baseCommand: [ls, -laR]
      stdout: lsout.txt
      inputs:
        dir:
          type: Directory
          inputBinding:
            position: 1
      outputs:
        ls_out:
          type: stdout
    in:
      dir: gen_dir/dir
    out: [ls_out]

outputs:
  ls_out:
    type: File
    outputSource: ls/ls_out

The workaround is to explicitly loop through the listing inside the input directory and map them using InitialWorkDirRequirement.

cwl • 274 views
ADD COMMENTlink modified 10 months ago by Tom530 • written 10 months ago by asha20
2
gravatar for Tom
10 months ago by
Tom530
Tom530 wrote:

I'm not able to test this right now, but my guess would be: Everything works as intended CWL-wise, meaning a symlink to the directory is created in the temporary directory where step two is carried out. Calling ls with the -lflag causes it to not follow symlinks, so nothing is listed.

Best regards, Tom

ADD COMMENTlink written 10 months ago by Tom530
1

Thank you so much, Tom! You are correct, the symlinks do indeed exist and show up with ls -LaR.

ADD REPLYlink written 10 months ago by asha20
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1764 users visited in the last hour