For loop script
1
0
Entering edit mode
5.1 years ago

Hi I am trying to run command in the for loop but it is not working. I have 49 file, of which i have to do multiplex and then align them afterwards. SO i thought it would be easy to do in for loop but i did not make loop script before, i tried to make this with help of friend but it seems there are some problem. Could someone help me to correct this? here is the script -

nohup sh -c 'for fq1 in *.fastq.gz; do
fq2=${fq1/_R1_/_R2_} out=basename$fq1 _L001_R1_001.fastq.gz
echo $fq1,$fq2, $out if [[ ! -f "$fq1" ]]; then echo "WRONG"; break; fi
if [[ ! -f "$fq2" ]]; then echo "WRONG"; break; fi fastq-multx -B barcode.txt$fq1 $fq2 -o${fq1%%.fastq.gz}_%.fa.gz; done

RNA-Seq fastq-multx • 2.7k views
3
Entering edit mode

Hi, please next time format your code to be better readable. Did you check my question - shell script to alignment paired-end reads?

2
Entering edit mode

Its a little hard to read your script since it is not formatted properly in the post, but something that immediately jumps out is that you are calling sh shell, but you are using [[ ... ]] for test, which I believe are bash specific and not supported in sh. Is there a reason you are running the command this way and not in a regular bash script?

2
Entering edit mode

I am trying to run command in the for loop but it is not working

What error are you getting? Are you getting no output file, but it appears to run? Remove nohup, and place your code into a bash script, with it broken up to make it easier to read. A one-liner can be difficult at first, and clunky to read.

for fq1 in *.fastq.gz; do

fq2=${fq1/_R1_/_R2_} out=basename$fq1 _L001_R1_001.fastq.gz

echo $fq1,$fq2, $out if [[ ! -f "$fq1" ]]; then
echo "WRONG"; break; fi

if [[ ! -f "$fq2" ]]; then echo "WRONG"; break; fi fastq-multx -B barcode.txt$fq1 $fq2 -o${fq1%%.fastq.gz}_%.fa.gz;
done

0
Entering edit mode

Error: number of input files (2) must match number of output files following '-o'.

0
Entering edit mode

Hi Steve, Thanks for comment. no reason for this, i got this script and run it. Could you suggest me to correct this?

0
Entering edit mode

Please use ADD COMMENT or ADD REPLY to answer to previous reactions, as such this thread remains logically structured and easy to follow. I have now moved your post but as you can see it's not optimal. Adding an answer should only be used for providing a solution to the question asked.

0
Entering edit mode

oh i see. I am new to this but now i will try to follow that.

0
Entering edit mode

Welcome to biostars. Interesting guidelines for posting can be found in the following posts:

0
Entering edit mode

but it is not working

I see.

seems there are some problem

Aha!

It would be very helpful if you can tell us what's going wrong. An error message, or an unexpected outcome.

0
Entering edit mode

hi Paul, I read you question and tried to use the suggested answer

for i in $(ls *.fastq.gz | rev | cut -c 13- | rev | uniq) > do > fastq-multx -B barcode.txt${i}_R1_001.fastq.gz ${i}_R2_001.fastq.gz -o${i%.fastq.gz}_%.fa.gz
> done


Error msg comes with following msg number of output should be same as input

0
Entering edit mode

Please use ADD COMMENT or ADD REPLY to answer to previous reactions, as such this thread remains logically structured and easy to follow. I have now moved your post but as you can see it's not optimal. Adding an answer should only be used for providing a solution to the question asked.

I added code markup to your post for increased readability. You can do this by selecting the text and clicking the 101010 button. When you compose or edit a post that button is in your toolbar, see image below:

3
Entering edit mode
5.1 years ago
steve ★ 3.3k

There are numerous issues with your script, you might want to spend some time checking out some of these tutorials brush some more syntax and formatting that will fix some of the problems.

I reformatted it to look a little better & should be closer to running (I can't test it myself);

#!/bin/bash

# iterating over all .fastq.gz files
for fq1 in *.fastq.gz; do
# create fastq 2 filename by subsituting "_R1_" in the filename for "_R2_"
fq2="${fq1/_R1_/_R2_}" # create an output file name by taking the basename of the input fastq file ?? out="$(basename ${fq1}_L001_R1_001.fastq.gz)" # you are probably going to need to specify the output directory as well, # gonna assume its the same as the input directory? # give it a new file extension? '.demult.fastq.gz' or something like that output_path="$(dirname ${fq1}_L001_R1_001.fastq.gz)/${out}.demult.fastq.gz"
echo "$fq1,$fq2, $out" # you are checking to make sure that both fastq1 and fastq2 exist... if [ ! -f "$fq1" ]; then
echo "WRONG"
break
fi
if [ ! -f "$fq2" ]; then echo "WRONG" break fi # I am not really sure what you meant by '${fq1%%.fastq.gz}_%.fa.gz' ...
fastq-multx -B barcode.txt "$fq1" "$fq2" -o "$output_path" done  However, there are still a lot of other issues with this script; since I love writing bash scripts, I reformatted the whole thing in a way which might work better: #!/bin/bash fastq_dir="$1"
output_dir="$2" barcode_file="barcode.txt" mkdir -p "$output_dir"

# find the R1 fastq files, e.g. 'fastq/sample1_L001_R1_001.fastq.gz'
find "$fastq_dir" -type f -name "*.fastq.gz" -name "*_R1_*" -print0 | while read -d$'\0' fastqR1; do
fastqR1_base="$(basename "$fastqR1")" # sample1_L001_R1_001.fastq.gz

# split the basename on the last '_R1_' and insert '_R2_'
fastqR2_base="${fastqR1_base%_R1_*}_R2_${fastqR1_base##*_R1_}" # sample1_L001_R2_001.fastq.gz

# make full path to the expected R2 fastq file
fastqR2="$(dirname "$fastqR1")/${fastqR2_base}" # make path to output file output_base="${fastqR1_base%_R1_*}_${fastqR1_base##*_R1_}" # sample1_L001_001.fastq.gz output_path="${output_dir}/${output_base}" # make sure fastqR2 exists if [ -f "$fastqR2" ] ; then
echo "Demultiplexing $fastqR1 and$fastqR2 with barcode file $barcode_file, output will be:$output_path"
fastq-multx -B "$barcode_file" "$fastqR1" "$fastqR2" -o "$output_path"
fi

done


Usage:

$ls -1 fastq/ sample1_L001_R1_001.fastq.gz sample1_L001_R2_001.fastq.gz sample2_L001_R1_001.fastq.gz sample2_L001_R2_001.fastq.gz sample3_L001_R1_001.fastq.gz$ ./demult2.sh fastq/ output
Demultiplexing fastq//sample1_L001_R1_001.fastq.gz and fastq/sample1_L001_R2_001.fastq.gz with barcode file barcode.txt, output will be: output/sample1_L001_001.fastq.gz
Demultiplexing fastq//sample2_L001_R1_001.fastq.gz and fastq/sample2_L001_R2_001.fastq.gz with barcode file barcode.txt, output will be: output/sample2_L001_001.fastq.gz


Hope that helps.

0
Entering edit mode

Hi Thanks for the comment. Here i am getting the following error msg.

Error: number of input files (2) must match number of output files following '-o'.

0
Entering edit mode
for fq1 in *.fastq.gz; do
fq2="${fq1/_R1_/_R2_}" out="$(basename ${fq1}_L001_R1_001.fastq.gz)" output_path="$(dirname ${fq1}_L001_R1_001.fastq.gz)/${out}_%.fa.gz"
echo "$fq1,$fq2, $out" if [[ ! -f "$fq1" ]]; then
echo "WRONG"
break
fi
if [[ ! -f "$fq2" ]]; then echo "WRONG" break fi fastq-multx -B barcode.txt "$fq1" "$fq2" -o "$output_path"
done
`

I ran the following script u suggested

0
Entering edit mode

There is nothing wrong with the script/loop. You are just using a wrong command for fastq-multx and the error message tells you exactly what's wrong. You have two input files ($fq1 and$fq2) so you need two output files.

I added code markup to your post for increased readability. You can do this by selecting the text and clicking the 101010 button. When you compose or edit a post that button is in your toolbar, see image below:

0
Entering edit mode

Thanks for comment! What is wrong with fastq-multx commond. Thanks

0
Entering edit mode

It just told you. Read the error message! It's one of the best error messages you'll ever see!

number of input files (2) must match number of output files following '-o'.

You have 2 input files. You need 2 output files. You only specified 1 output file.

0
Entering edit mode

Thanks that I got it. I thought u ment something else. Thanks