Mixing CWL workflow versions (including draft-3 tool in v1.0 workflow)
1
1
Entering edit mode
8.2 years ago
Peter vH ▴ 130

I'm working on a CWL v1.0 workflow, using the GGR-cwl trimmomatic tool (https://github.com/Duke-GCB/GGR-cwl/blob/master/trimmomatic/trimmomatic.cwl) as one of the steps. That tool is a draft-3 specification one, which I understand is still parsed by cwltool.

Here is my current workflow file:

class: Workflow
requirements:
  - class: StepInputExpressionRequirement
inputs:
  reads1: File
  reads2: File
  slidingw:
    type: string
    default: "30"
  minl:
    type: int
    default: 20
outputs:
  trimmed_reads:
    type: File
    outputSource: trim_reads/output_read1_trimmed_file

steps:
  trim_reads:
    run: { $import: "../../tools/trimmomatic/trimmomatic.cwl" }
    in:
      input_read1_fastq_file: reads1
      input_read2_fastq_file: reads2
      slidingwindow: slidingw
      minlen: minl
      end_mode:
        valueFrom: "PE"


    out: [output_read1_trimmed_file]

This, however, fails to validate, I think because the $import results in mixing draft-3 and v1.0 CWL. Is there any way to use draft-3 CWL with v1.0?

More detail:

The errors are extensive but start as follows:

/cip0/software/x86_64/miniconda-3.19.0/bin/cwltool 1.0.20160714182449
Tool definition failed validation:
Validation error in object file:///net/ceph-mon1.sanbi.ac.za/sanbi/scratch/pvh/cwl/workflows/trim/trim.cwl
  Could not validate as `CommandLineTool` because
    could not validate field `outputs` because
      At position 0
        could not validate field `outputSource` because it is not recognized and strict is True, valid fields are: label, secondaryFiles, format, streamable, doc, id, outputBinding, type

    missing required field `baseCommand`
    could not validate field `steps` because it is not recognized and strict is True, valid fields are: id, inputs, outputs, requirements, hints, label, doc, cwlVersion, class, baseCommand, arguments, stdin, stderr, stdout, successCodes, temporaryFailCodes, permanentFailCodes
  Could not validate as `ExpressionTool` because
    could not validate field `outputs` because
      At position 0
        could not validate field `outputSource` because it is not recognized and strict is True, valid fields are: label, secondaryFiles, format, streamable, doc, id, outputBinding, type

    missing required field `expression`
    could not validate field `steps` because it is not recognized and strict is True, valid fields are: id, inputs, outputs, requirements, hints, label, doc, cwlVersion, class, expression
  Could not validate as `Workflow` because
    could not validate field `steps` because
      the value `[{'id': 'file:///net/ceph-mon1.sanbi.ac.za/sanbi/scratch/pvh/cwl/workflows/trim/trim.cwl#trim_reads'
        'in': [{'id': u'file:///net/ceph-mon1.sanbi.ac.za/sanbi/[...]`
       is not a valid type in the union, expected one of:
      - array of <WorkflowStep>, but
         At position 0
          could not validate field `run` because
            the value `{'arguments': [{'position': 1,
                            'shellQuote': False,
                            'valueFrom': '$("-Djava.io.tmpdir="+runtime.tmpdir)'},
                           {'posi[...]`
             is not a valid type in the union, expected one of:
            - string, but
               the value `{'arguments': [{'position': 1,
                              'shellQuote': False,
                              'valueFrom': '$("-Djava.io.tmpdir="+runtime.tmpdir)'},
                             {'posi[...]` is not string
            - CommandLineTool, but
               could not validate field `inputs` because
                At position 0
                  could not validate field `description` because it is not recognized and strict is True, valid fields are: label, secondaryFiles, format, streamable, doc, id, inputBinding, default, type

              could not validate field `outputs` because
                At position 4
                  could not validate field `description` because it is not recognized and strict is True, valid fields are: label, secondaryFiles, format, streamable, doc, id, outputBinding, type

              could not validate field `description` because it is not recognized and strict is True, valid fields are: id, inputs, outputs, requirements, hints, label, doc, cwlVersion, class, baseCommand, arguments, stdin, stderr, stdout, successCodes, temporaryFailCodes, permanentFailCodes

Note the reference to field description that is valid in draft-3 but not v1.0. The equivalent draft-3 workflow seems to be accepted as valid:

cwlVersion: cwl:draft-3
class: Workflow
requirements:
  - class: StepInputExpressionRequirement

inputs: 
  - id: reads1
    type: File
  - id: reads2
    type: File
  - id: slidingw
    type: string
    default: "30"
  - id: minl
    type: int
    default: 20
outputs:
  - id: trimmed_reads
    type: File
    source: "#trim_reads/output_read1_trimmed_file"

steps:
  - id: trim_reads
    run: { $import: "../../tools/trimmomatic/trimmomatic.cwl" }
    inputs:
      - id: input_read1_fastq_file
        source: "#reads1"
      - id: input_read2_fastq_file 
        source: "#reads2"
      - id: slidingwindow
        source: "#slidingw"
      - id: minlen
        source: "#minl"
      - id: end_mode
        valueFrom: "PE"
    outputs: 
      - id: output_read1_trimmed_file
cwl • 2.2k views
ADD COMMENT
0
Entering edit mode

For interest's sake, here is a single-step draft-3 workflow that works:

cwlVersion: cwl:draft-3
class: Workflow
requirements:
  - class: StepInputExpressionRequirement

inputs: 
  - id: reads1
    type: File
  - id: reads2
    type: File
  - id: slidingw
    type: string
    default: "4:30"
  - id: minl
    type: int
    default: 20
outputs:
  - id: trimmed_reads
    type: File
    source: "#trim_reads.output_read1_trimmed_file"

steps:
  - id: trim_reads
    run: { $import: "../../tools/trimmomatic/trimmomatic.cwl" }
    inputs:
      - id: input_read1_fastq_file
        source: "#reads1"
      - id: input_read2_fastq_file 
        source: "#reads2"
      - id: slidingwindow
        source: "#slidingw"
      - id: minlen
        source: "#minl"
      - id: end_mode
        valueFrom: "PE"
      - id: trimmomatic_jar_path
        valueFrom: "/cip0/software/java/Trimmomatic-0.33/trimmomatic-0.33.jar"
    outputs: 
      - id: "#trim_reads.output_read1_trimmed_file"
ADD REPLY
2
Entering edit mode
8.2 years ago

The run: { $import ... } idiom is from the draft-2 era. draft-3 onwards you can just give the path to the CWL tool description.

cwlVersion: v1.0
[...]
steps:
  trim_reads:
    run: "../../tools/trimmomatic/trimmomatic.cwl"
ADD COMMENT
1
Entering edit mode

Thanks! It works now, here's the final single-step v1.0 workflow:

cwlVersion: v1.0
class: Workflow
requirements:
  - class: StepInputExpressionRequirement
inputs:
  reads1: File
  reads2: File
  slidingw:
    type: string
    default: "4:30"
  minl:
    type: int
    default: 20
outputs:
  trimmed_reads:
    type: File
    outputSource: trim_reads/output_read1_trimmed_file

steps:
  trim_reads:
    run: "../../tools/trimmomatic/trimmomatic.cwl" 
    in:
      input_read1_fastq_file: reads1
      input_read2_fastq_file: reads2
      slidingwindow: slidingw
      minlen: minl
      end_mode:
        valueFrom: "PE"
      trimmomatic_jar_path:
        valueFrom: "/cip0/software/java/Trimmomatic-0.33/trimmomatic-0.33.jar"

    out: [output_read1_trimmed_file]
ADD REPLY

Login before adding your answer.

Traffic: 1147 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6