Question: CWL: Getting the outputs(File) of a command line tool
0
gravatar for kevin.o.oluoch
23 months ago by
Kenya
kevin.o.oluoch70 wrote:

I've executed the following Command Line Tool.

#!/usr/bin/env cwl-runner
cwlVersion: v1.0
class: CommandLineTool

baseCommand: spades.py
requirements:
  - class: ShellCommandRequirement

inputs:

  SAMPLENAME: string
  OUTPUT_DIR: string

  FORWARD: 
    type: string
    inputBinding:
      position: 1
      prefix: '-1'

  REVERSE: 
    type: string
    inputBinding:
      position: 2
      prefix: '-2'

  THREADS: 
    type: int
    inputBinding:
      position: 4
      prefix: "--threads"
      shellQuote: false

outputs:
  OUTPUT_FILE:
    type: File
    outputBinding:
      glob: "contigs.fasta"

arguments:
  - valueFrom: $(inputs.OUTPUT_DIR)/$(inputs.SAMPLENAME) 
    position: 3
    prefix: "-o "
    shellQuote: false

The command on the terminal is:

./spadesTool.cwl  --SAMPLENAME="11866_1#70" --FORWARD="/media/kevin/2251479E904444F7/data/11866_1#70_1.fastq.gz" --REVERSE="/media/kevin/2251479E904444F7/data/11866_1#70_2.fastq.gz" --OUTPUT_DIR="/media/kevin/2251479E904444F7/Output_Directory" --THREADS=4

The baseCommand spades.py) executes successfully and produces results. However CWL produces the following error

[ job spadesTool.cwl ] Job error :
spadesTool.cwl:38:3: Error collecting output for parameter "OUTPUT_FILE" :
spadesTool.cwl:41:7: Did not find output file with glob pattern: '[' contigs.fasta ']' 
[ job spadesTool.cwl ] completed permanentFail
{}
Final process status is permanentFail

What could be the problem?

ADD COMMENTlink modified 22 months ago by Michael R. Crusoe1.2k • written 23 months ago by kevin.o.oluoch70

Why not just copy it to another directory?

$cd Desktop

$cp OUTPUT_DIR/contigs.fasta .

ADD REPLYlink modified 23 months ago • written 23 months ago by YaGalbi1.4k

The file and its parent directory will be created by the tool. I need the file to be capture as output of the Command Line TOOL, using the variable OUTPUT_FILE

ADD REPLYlink written 23 months ago by kevin.o.oluoch70

Post the command you are using please

It may be as simple as using a pipe.

ADD REPLYlink written 23 months ago by YaGalbi1.4k
./spadesTool.cwl  --SAMPLENAME="11866_1#70" --FORWARD="/media/kevin/2251479E904444F7/data/11866_1#70_1.fastq.gz" --REVERSE="/media/kevin/2251479E904444F7/data/11866_1#70_2.fastq.gz" --OUTPUT_DIR="/media/kevin/2251479E904444F7/Output_Directory" --THREADS=4

I'll do this for several samples, and I want the results as sub-directories, in the output directory.

ADD REPLYlink written 22 months ago by kevin.o.oluoch70

Well now you mention sub-directories...If you are just running this in the command line then I suggest a for loop. Either in the command line or in a script. You would need to store a list of sample names for iterating through. then on each iteration create a new folder (named same as sample name)... use the folder as the output folder

something like this in command line should work (excluding minor synthax errors):

for i in SAMPLE_LIST; do mkdir $i; ./spadesTool.cwl  --SAMPLENAME=$i --FORWARD="/media/kevin/2251479E904444F7/data/$i_1.fastq.gz" --REVERSE="/media/kevin/2251479E904444F7/data/$i_2.fastq.gz" --OUTPUT_DIR=">/media/kevin/2251479E904444F7/$i" --THREADS=4; done

If you follow the idea then a folder will be created for each sample containing only outfiles for that sample.

ADD REPLYlink modified 22 months ago • written 22 months ago by YaGalbi1.4k

This does not solve my original issue. Kindly see the change I've made to the question

ADD REPLYlink written 22 months ago by kevin.o.oluoch70

Ok this is starting to sound like a job for TEE

sorry I can't be of more help

Yes Ram thank you :)

ADD REPLYlink written 22 months ago by YaGalbi1.4k

makedir? Do you mean mkdir?

ADD REPLYlink written 22 months ago by RamRS20k
2
gravatar for Michael R. Crusoe
22 months ago by
currently based out of Vilnius, Lithuania / Common Workflow Language project
Michael R. Crusoe1.2k wrote:

Hello kevin.o.oluoch,

Thank you for your question.

In CWL it is incorrect to pass a path to a local directory as a string for a variety of reasons. In this instance you are trying to communicate to spades where to write its output -- since CWL tools could be executed remotely it would be simpler to use the empty working directory one is guaranteed to have available by the CWL spec:

arguments:
  - valueFrom: $(runtime.outdir)
    prefix: -o

FYI: You may be interested in the CWL description I made for SPAdes operating in metaSPAdes mode:

https://github.com/ProteinsWebTeam/ebi-metagenomics-cwl/blob/master/tools/metaspades.cwl

Feel free to copy and modify it under the terms of the Apache 2 license.

ADD COMMENTlink written 22 months ago by Michael R. Crusoe1.2k

Thanks, This is very Helpful

ADD REPLYlink written 22 months ago by kevin.o.oluoch70
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1216 users visited in the last hour