Question: Using VelvetOptimiser.pl within a forloop
1
gravatar for c.e.chong
29 days ago by
c.e.chong10
c.e.chong10 wrote:

Hi,

I am trying to run VelvetOptimiser on a number of paired end read files.

I have installed both velvet and velvetoptimiser via Conda. I wrote this forloop to loop through all of my read fliles:

for i in *_1_*.fastq; do VelvetOptimiser.pl -t 12 -s 19 -v --d ~/comp_genomics/velvet_optimiser_jo/${i%%_1_*.fastq}_assembly -f '-fastq -shortPaired $i ${i%%_1_*.fastq}_2_*.fastq'; done

I am however getting this error message:

Will run velvet optimiser with the following paramters:
Velveth parameter string:
    -fastq -shortPaired $i ${i%%_1_*.fastq}_2_*.fastq
Velveth start hash values:  19
Velveth end hash value:     191
Velveth hash step value:    2
Velvetg minimum coverage cutoff to use: 0

Read tracking for final assembly off. Jan 17 16:41:17 Beginning velveth runs. Jan 17 16:41:17       Running velveth with hash value: 19. velveth: Could not open _2_*.fastq: No such file or directory Velveth failed!  Response:1

I have tested the separate components of the command not in the forloop. So I know that separately the loop works as does the command options.

for i in *_1_*.fastq; do echo '-fastq -shortPaired' $i ${i%%_1_*.fastq}_2_*.fastq; done


VelvetOptimiser.pl -t 12 -s 19 -v --d ~/comp_genomics/velvet_optimiser_jo/bb_17198wB1_Sample_2_assembly -f '-shortPaired -fastq bb_17198wB1_Sample_2_1_trimmed_prinseq.fastq bb_17198wB1_Sample_2_2_trimmed_prinseq.fastq'

Is it possible to use VelvetOptimiser in a forloop? Is there a mistake that I'm missing?

I'd be very grateful for any help!

Thank you in advance!!!

sequencing velvet assembly • 161 views
ADD COMMENTlink written 29 days ago by c.e.chong10
1

How come the output of

for i in *_1_*.fastq; do echo '-fastq -shortPaired' $i ${i%%_1_*.fastq}_2_*.fastq; done

is

VelvetOptimiser.pl -t 12 -s 19 -v --d ~/comp_genomics/velvet_optimiser_jo/bb_17198wB1_Sample_2_assembly -f '-shortPaired -fastq bb_17198wB1_Sample_2_1_trimmed_prinseq.fastq bb_17198wB1_Sample_2_2_trimmed_prinseq.fastq'

This is simply not possible.

In addition, explain your files naming convention, so we can better understand your problem. If all files are named {something}_trimmed_prinseq.fastq, I would use for i in *_1_trimmed_prinseq.fastq. If you pay attention, *_1_*.fastq will potentially match a file named bb_17198wB1_Sample_1_2_trimmed_prinseq.fastq, in case there is one.

ADD REPLYlink written 28 days ago by h.mon23k
VelvetOptimiser.pl -t 12 -s 19 -v --d ~/comp_genomics/velvet_optimiser_jo/bb_17198wB1_Sample_2_assembly -f '-shortPaired -fastq bb_17198wB1_Sample_2_1_trimmed_prinseq.fastq bb_17198wB1_Sample_2_2_trimmed_prinseq.fastq'

is not the output of

for i in *_1_*.fastq; do echo '-fastq -shortPaired' $i ${i%%_1_*.fastq}_2_*.fastq; done

I was simply showing the two commands I ran to test both the velvet command itself and the the forloop.

An example of my file names is bb_17198wD7_Sample_52_1_trimmed_prinseq.fastq, I can take out the second wild card incase it would call the wrong file. But I haven't had this problem so far.

Thanks

ADD REPLYlink written 25 days ago by c.e.chong10
4
gravatar for jrj.healey
29 days ago by
jrj.healey10k
United Kingdom
jrj.healey10k wrote:

By putting this section:

'-fastq -shortPaired $i ${i%%_1_*.fastq}_2_*.fastq'

in single quotes, you've hard quoted it, which means the variables aren't interpreted, its just treated as a string literal. You need to use ".

Also, Velvet is pretty old news at this point - is there a particular reason to use it in your case?

ADD COMMENTlink modified 29 days ago • written 29 days ago by jrj.healey10k

Thanks for getting back to me!

I tried using " but this didn't work either. The input read files have to be within a ' to call Velveth.

I understand that Velvet is old news now, I was just trying to carry out some comparison assemblies as my group used to use velvet.

ADD REPLYlink written 28 days ago by c.e.chong10
1

The fact that velvet spits out:

Will run velvet optimiser with the following paramters:
Velveth parameter string:
    -fastq -shortPaired $i ${i%%_1_*.fastq}_2_*.fastq

Tells me that your variables aren’t being expanded so that must be at least part of the problem. Do you get the same error when you replace the ‘ with “?

ADD REPLYlink written 28 days ago by jrj.healey10k

When I replace the ' with " I get this error

Number of CPUs available: 64
Current free RAM: 496.545GB
Velvet OMP compiler setting: 1
Unknown option: shortpaired bb_17198wa9_sample_65_1_trimmed_prinseq.fastq bb_17198wa9_sample_65_2_*.fastq
Usage: /pub38/cchong/miniconda2/envs/velvet_env/bin/VelvetOptimiser.pl [options] -f 'velveth input line'

So this does call the correct files but this causes velveth not to run!

Thanks for your help!

ADD REPLYlink modified 25 days ago by jrj.healey10k • written 25 days ago by c.e.chong10

Your filenames are expanding with spaces in which Velvet is interpreting as multiple unknown arguments.

If I do:

$ ls *.fastq.gz*
A_S1_L001_R1_001.fastq.gz  A_S1_L001_R2_001.fastq.gz

I have 2 dummy fastq files here, then:

for i in *_L001_* ; do echo "VelvetOptimiser.pl -t 12 -s 19 -v --d ~/comp_genomics/velvet_optimiser_jo/${i%%_1_*.fastq}_assembly -f -fastq -shortPaired $i ${i%%_1_*.fastq}_2_*.fastq" ; done

Gives:

VelvetOptimiser.pl -t 12 -s 19 -v --d ~/comp_genomics/velvet_optimiser_jo/A_S1_L001_R1_001.fastq.gz_assembly -f -fastq -shortPaired A_S1_L001_R1_001.fastq.gz A_S1_L001_R1_001.fastq.gz_2_*.fastq
VelvetOptimiser.pl -t 12 -s 19 -v --d ~/comp_genomics/velvet_optimiser_jo/A_S1_L001_R2_001.fastq.gz_assembly -f -fastq -shortPaired A_S1_L001_R2_001.fastq.gz A_S1_L001_R2_001.fastq.gz_2_*.fastq

There are multiple issues here. Firstly, you cannot just throw wildcards (*) in where ever you like. Particularly your second read file is specified totally wrong (${i%%_1_*.fastq}_2_*.fastq).

You need to start over and build the command up bit at a time, until you're certain that the variables expand correctly.

ADD REPLYlink written 25 days ago by jrj.healey10k

Thank you all for your help I have resolved the issue now, using extra '.

'-shortPaired -fastq '$i' '${i%%_1_trimmed_prinseq.fastq}_2_trimmed_prinseq.fastq''
ADD REPLYlink written 25 days ago by c.e.chong10
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 756 users visited in the last hour