Question: CWL: Getting the outputs(File) of a command line tool
0
gravatar for kevin.o.oluoch
2.3 years ago by
Kenya
kevin.o.oluoch70 wrote:

I've executed the following Command Line Tool.

#!/usr/bin/env cwl-runner
cwlVersion: v1.0
class: CommandLineTool

baseCommand: spades.py
requirements:
  - class: ShellCommandRequirement

inputs:

  SAMPLENAME: string
  OUTPUT_DIR: string

  FORWARD: 
    type: string
    inputBinding:
      position: 1
      prefix: '-1'

  REVERSE: 
    type: string
    inputBinding:
      position: 2
      prefix: '-2'

  THREADS: 
    type: int
    inputBinding:
      position: 4
      prefix: "--threads"
      shellQuote: false

outputs:
  OUTPUT_FILE:
    type: File
    outputBinding:
      glob: "contigs.fasta"

arguments:
  - valueFrom: $(inputs.OUTPUT_DIR)/$(inputs.SAMPLENAME) 
    position: 3
    prefix: "-o "
    shellQuote: false

The command on the terminal is:

./spadesTool.cwl  --SAMPLENAME="11866_1#70" --FORWARD="/media/kevin/2251479E904444F7/data/11866_1#70_1.fastq.gz" --REVERSE="/media/kevin/2251479E904444F7/data/11866_1#70_2.fastq.gz" --OUTPUT_DIR="/media/kevin/2251479E904444F7/Output_Directory" --THREADS=4

The baseCommand spades.py) executes successfully and produces results. However CWL produces the following error

[ job spadesTool.cwl ] Job error :
spadesTool.cwl:38:3: Error collecting output for parameter "OUTPUT_FILE" :
spadesTool.cwl:41:7: Did not find output file with glob pattern: '[' contigs.fasta ']' 
[ job spadesTool.cwl ] completed permanentFail
{}
Final process status is permanentFail

What could be the problem?

ADD COMMENTlink modified 2.3 years ago by Michael R. Crusoe1.6k • written 2.3 years ago by kevin.o.oluoch70

Why not just copy it to another directory?

$cd Desktop

$cp OUTPUT_DIR/contigs.fasta .

ADD REPLYlink modified 2.3 years ago • written 2.3 years ago by YaGalbi1.4k

The file and its parent directory will be created by the tool. I need the file to be capture as output of the Command Line TOOL, using the variable OUTPUT_FILE

ADD REPLYlink written 2.3 years ago by kevin.o.oluoch70

Post the command you are using please

It may be as simple as using a pipe.

ADD REPLYlink written 2.3 years ago by YaGalbi1.4k
./spadesTool.cwl  --SAMPLENAME="11866_1#70" --FORWARD="/media/kevin/2251479E904444F7/data/11866_1#70_1.fastq.gz" --REVERSE="/media/kevin/2251479E904444F7/data/11866_1#70_2.fastq.gz" --OUTPUT_DIR="/media/kevin/2251479E904444F7/Output_Directory" --THREADS=4

I'll do this for several samples, and I want the results as sub-directories, in the output directory.

ADD REPLYlink written 2.3 years ago by kevin.o.oluoch70

Well now you mention sub-directories...If you are just running this in the command line then I suggest a for loop. Either in the command line or in a script. You would need to store a list of sample names for iterating through. then on each iteration create a new folder (named same as sample name)... use the folder as the output folder

something like this in command line should work (excluding minor synthax errors):

for i in SAMPLE_LIST; do mkdir $i; ./spadesTool.cwl  --SAMPLENAME=$i --FORWARD="/media/kevin/2251479E904444F7/data/$i_1.fastq.gz" --REVERSE="/media/kevin/2251479E904444F7/data/$i_2.fastq.gz" --OUTPUT_DIR=">/media/kevin/2251479E904444F7/$i" --THREADS=4; done

If you follow the idea then a folder will be created for each sample containing only outfiles for that sample.

ADD REPLYlink modified 2.3 years ago • written 2.3 years ago by YaGalbi1.4k

This does not solve my original issue. Kindly see the change I've made to the question

ADD REPLYlink written 2.3 years ago by kevin.o.oluoch70

Ok this is starting to sound like a job for TEE

sorry I can't be of more help

Yes Ram thank you :)

ADD REPLYlink written 2.3 years ago by YaGalbi1.4k

makedir? Do you mean mkdir?

ADD REPLYlink written 2.3 years ago by RamRS23k
2
gravatar for Michael R. Crusoe
2.3 years ago by
Common Workflow Language project
Michael R. Crusoe1.6k wrote:

Hello kevin.o.oluoch,

Thank you for your question.

In CWL it is incorrect to pass a path to a local directory as a string for a variety of reasons. In this instance you are trying to communicate to spades where to write its output -- since CWL tools could be executed remotely it would be simpler to use the empty working directory one is guaranteed to have available by the CWL spec:

arguments:
  - valueFrom: $(runtime.outdir)
    prefix: -o

FYI: You may be interested in the CWL description I made for SPAdes operating in metaSPAdes mode:

https://github.com/ProteinsWebTeam/ebi-metagenomics-cwl/blob/master/tools/metaspades.cwl

Feel free to copy and modify it under the terms of the Apache 2 license.

ADD COMMENTlink written 2.3 years ago by Michael R. Crusoe1.6k

Thanks, This is very Helpful

ADD REPLYlink written 2.3 years ago by kevin.o.oluoch70
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1913 users visited in the last hour