error when running NGS tools in bash script
0
0
Entering edit mode
20 months ago
bertb ▴ 10

Hello,

I am trying to write a script to automate my RNAseq pipeline in a Bash shell on Ubuntu. Very simply, the beginning of my script (written in Vim) looks like this:

#!/bin/bash
set -e
set -u
set -o pipefail

#set variable for samples text file
sample_info=samples.txt

#set variable for fastq files
fq=($(cut -f 3 "$sample_info"))

for fastqc_file in ${fq[@]} do fastqc$fastqc_file
done


and the error I receive is

Skipping '~/Documents/bash_practice/tutorial/UMH_R1a.fastq' which didn't exist, or couldn't be read


The samples.txt is a simple file with 3 tab separated columns containing name, read #, and file path:

UMH R1  ~/Documents/bash_practice/tutorial/UMH_R1a.fastq
UMH R2  ~/Documents/bash_practice/tutorial/UMH_R2a.fastq
UMN R1  ~/Documents/bash_practice/tutorial/UMN_R1a.fastq
UMN R2  ~/Documents/bash_practice/tutorial/UMN_R2a.fastq


I know there is no issue with the file or location because when I run:

fastqc ~/Documents/bash_practice/tutorial/UMH_R1a.fastq


on its own, everything runs fine. So I think the issue must be with how the script is accessing or trying to process my samples.txt file.

I am new to writing scripts, so the answer may be something trivial, and my apologies if it is very obvious.

software error automation scripting • 577 views
0
Entering edit mode

Arrays are kind of unnecessary here. Given that arrays are almost abusive to the shell, I don't see any merit to this approach. Why not a simple

for fq_file in $(cut -f3$sample_info)
do
fastqc $fq_file done  or even better, use GNU parallel. I can't type the command off the top of my head, but it should be pretty straightforward to create. ADD REPLY 0 Entering edit mode Hi, Thanks for the reply. retyped the script exactly as you suggested, and I still receive the same error. When I tell the script to just execute the command on a particular file, it works fine. It seems to be something about it retrieving it from the text file that is an issue. Any suggestions? Thanks, ADD REPLY 0 Entering edit mode Can you try changing the ~ to the absolute path to your home directory in the text file? If your home directory is /home/bertb, try: sed -i 's#~#/home/bertb#'$sample_info


Maybe the shell is seeing ~ as a literal and not as a link to your home directory.

0
Entering edit mode

one way to troubleshoot bash scripts is to put an echo in front of your command

echo fastqc \$fastqc_file


now it prints the commands, save that into a file and execute those with bash see if it really comes out the way you think it should

 bash myscripts > commands.txt


look at your commands, now execute them:

 bash commands.txt