Question: CWL Error Empty Scatter
0
gravatar for HL
14 months ago by
HL0
HL0 wrote:

Whenever I try do scatter over an array of files I get the error message

emptyscatter = [shortname(s) for s in scatter if len(inputobj[s]) == 0]
TypeError: object of type 'NoneType' has no len()

This happens only, if I use a JavaScript expression to concatenate the arrays, which I have to do in this case. If I use only one of the arrays without JS expression the scatter works fine.

Does anyone have an idea how to fix this? Please see my code below:

The Workflow File

cwlVersion: v1.0
class: Workflow

requirements:
  - class: ScatterFeatureRequirement
  - class: SubworkflowFeatureRequirement
  - class: StepInputExpressionRequirement
  - class: InlineJavascriptRequirement

inputs:

  arrayC:
    type:
    - type: array
      items: File
    secondaryFiles:
      - ".bai"

  arrayD:
    type:
    - type: array
      items: File
    secondaryFiles:
      - ".bai"

  arrayE: string[]
  arrayF: string[]

outputs: []

steps:
  stepname:
    run: <path to CWL>
    scatter: [A, B]
    scatterMethod: dotproduct
    in:
      A:
        valueFrom: $( return arrayC.concat(arrayD))
      B:
        valueFrom: $( return arrayE.concat(arrayF).map(function(e) {return e + ".raw.bamcov"}))

      arrayC: arrayC
      arrayD: arrayD
      arrayE: arrayE
      arrayF: arrayF

    out:
      - outputFile

The .yml File

arrayC: [
  {class: File, path: testfiles/file_1.bam},
  {class: File, path: testfiles/file_2.bam}
]

arrayD: [
  {class: File, path: testfiles/file_3.bam},
  {class: File, path: testfiles/file_4.bam}
]

arrayE: ["file_name_1", "file_name_2"]

arrayF: ["file_name_3", "file_name_4"]
scatter pipeline workflow cwl • 467 views
ADD COMMENTlink written 14 months ago by HL0

Hello HL, thank you for your question.

$( return arrayC.concat(arrayD)) is not a valid expression. Did you mean $( return inputs.arrayC.concat(inputs.arrayD))?

Can you provide a testable example so we can help you?

ADD REPLYlink written 14 months ago by Michael R. Crusoe1.8k

For A you can accomplish this using linkMerge: merge_flattened:

steps:
  stepname:
    run: <path to CWL>
    scatter: [A, B]
    scatterMethod: dotproduct
    in:
      A:
        source: arrayC, arrayD
        linkMerge: merge_flattened
ADD REPLYlink written 14 months ago by Michael R. Crusoe1.8k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 2001 users visited in the last hour