Missing CWL files in a Directory when passed as input

Question

[asha]

I wasn't sure if this was a question or a bug, so decided to first post a question about it. I can file an issue if it's confirmed to be a bug.

It looks like files are not mounted properly in directories when they're passed from one tool to another. Note that the files are tracked properly by CWL; they're just not mapped to the directory.

An example workflow is shown below. The output of ls is just an empty directory. The expected output is a full list of files and their subdirectories. I ran it using cwl-runner v1.0 and cwltool v1.0.20190831161204. Is there something that I'm missing when operating on directories in CWL?

class: Workflow
cwlVersion: v1.0
requirements:
  InlineJavascriptRequirement: {}
  SubworkflowFeatureRequirement: {}

inputs: {}
steps:
  gen_dir:
    run:
      class: CommandLineTool
      baseCommand: [bash, -c, 'mkdir -p test/subdir; echo hello > test/hello.txt; echo world > test/subdir/world.txt']
      inputs: {}
      outputs:
        dir:
          type: Directory
          outputBinding:
            glob: 'test'
    in: {}
    out: [dir]

  ls:
    run:
      class: CommandLineTool
      baseCommand: [ls, -laR]
      stdout: lsout.txt
      inputs:
        dir:
          type: Directory
          inputBinding:
            position: 1
      outputs:
        ls_out:
          type: stdout
    in:
      dir: gen_dir/dir
    out: [ls_out]

outputs:
  ls_out:
    type: File
    outputSource: ls/ls_out

The workaround is to explicitly loop through the listing inside the input directory and map them using InitialWorkDirRequirement.

Answer 1

[Tom]

I'm not able to test this right now, but my guess would be: Everything works as intended CWL-wise, meaning a symlink to the directory is created in the temporary directory where step two is carried out. Calling ls with the -lflag causes it to not follow symlinks, so nothing is listed.

Best regards, Tom

[asha]

Thank you so much, Tom! You are correct, the symlinks do indeed exist and show up with ls -LaR.