CWL: Getting the outputs(File) of a command line tool
1
0
Entering edit mode
5.4 years ago

I've executed the following Command Line Tool.

#!/usr/bin/env cwl-runner
cwlVersion: v1.0
class: CommandLineTool

baseCommand: spades.py
requirements:
  - class: ShellCommandRequirement

inputs:

  SAMPLENAME: string
  OUTPUT_DIR: string

  FORWARD: 
    type: string
    inputBinding:
      position: 1
      prefix: '-1'

  REVERSE: 
    type: string
    inputBinding:
      position: 2
      prefix: '-2'

  THREADS: 
    type: int
    inputBinding:
      position: 4
      prefix: "--threads"
      shellQuote: false

outputs:
  OUTPUT_FILE:
    type: File
    outputBinding:
      glob: "contigs.fasta"

arguments:
  - valueFrom: $(inputs.OUTPUT_DIR)/$(inputs.SAMPLENAME) 
    position: 3
    prefix: "-o "
    shellQuote: false

The command on the terminal is:

./spadesTool.cwl  --SAMPLENAME="11866_1#70" --FORWARD="/media/kevin/2251479E904444F7/data/11866_1#70_1.fastq.gz" --REVERSE="/media/kevin/2251479E904444F7/data/11866_1#70_2.fastq.gz" --OUTPUT_DIR="/media/kevin/2251479E904444F7/Output_Directory" --THREADS=4

The baseCommand spades.py) executes successfully and produces results. However CWL produces the following error

[ job spadesTool.cwl ] Job error :
spadesTool.cwl:38:3: Error collecting output for parameter "OUTPUT_FILE" :
spadesTool.cwl:41:7: Did not find output file with glob pattern: '[' contigs.fasta ']' 
[ job spadesTool.cwl ] completed permanentFail
{}
Final process status is permanentFail

What could be the problem?

cwl outputs Common Workflow Language • 3.9k views
ADD COMMENT
0
Entering edit mode

Why not just copy it to another directory?

$cd Desktop

$cp OUTPUT_DIR/contigs.fasta .

ADD REPLY
0
Entering edit mode

The file and its parent directory will be created by the tool. I need the file to be capture as output of the Command Line TOOL, using the variable OUTPUT_FILE

ADD REPLY
0
Entering edit mode

Post the command you are using please

It may be as simple as using a pipe.

ADD REPLY
0
Entering edit mode
./spadesTool.cwl  --SAMPLENAME="11866_1#70" --FORWARD="/media/kevin/2251479E904444F7/data/11866_1#70_1.fastq.gz" --REVERSE="/media/kevin/2251479E904444F7/data/11866_1#70_2.fastq.gz" --OUTPUT_DIR="/media/kevin/2251479E904444F7/Output_Directory" --THREADS=4

I'll do this for several samples, and I want the results as sub-directories, in the output directory.

ADD REPLY
0
Entering edit mode

Well now you mention sub-directories...If you are just running this in the command line then I suggest a for loop. Either in the command line or in a script. You would need to store a list of sample names for iterating through. then on each iteration create a new folder (named same as sample name)... use the folder as the output folder

something like this in command line should work (excluding minor synthax errors):

for i in SAMPLE_LIST; do mkdir $i; ./spadesTool.cwl  --SAMPLENAME=$i --FORWARD="/media/kevin/2251479E904444F7/data/$i_1.fastq.gz" --REVERSE="/media/kevin/2251479E904444F7/data/$i_2.fastq.gz" --OUTPUT_DIR=">/media/kevin/2251479E904444F7/$i" --THREADS=4; done

If you follow the idea then a folder will be created for each sample containing only outfiles for that sample.

ADD REPLY
0
Entering edit mode

This does not solve my original issue. Kindly see the change I've made to the question

ADD REPLY
0
Entering edit mode

Ok this is starting to sound like a job for TEE

sorry I can't be of more help

Yes Ram thank you :)

ADD REPLY
0
Entering edit mode

makedir? Do you mean mkdir?

ADD REPLY
2
Entering edit mode
5.4 years ago

Hello kevin.o.oluoch,

Thank you for your question.

In CWL it is incorrect to pass a path to a local directory as a string for a variety of reasons. In this instance you are trying to communicate to spades where to write its output -- since CWL tools could be executed remotely it would be simpler to use the empty working directory one is guaranteed to have available by the CWL spec:

arguments:
  - valueFrom: $(runtime.outdir)
    prefix: -o

FYI: You may be interested in the CWL description I made for SPAdes operating in metaSPAdes mode:

https://github.com/ProteinsWebTeam/ebi-metagenomics-cwl/blob/master/tools/metaspades.cwl

Feel free to copy and modify it under the terms of the Apache 2 license.

ADD COMMENT
0
Entering edit mode

Thanks, This is very Helpful

ADD REPLY

Login before adding your answer.

Traffic: 844 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6