error when running NGS tools in bash script
0
0
Entering edit mode
20 months ago
bertb ▴ 10

Hello,

I am trying to write a script to automate my RNAseq pipeline in a Bash shell on Ubuntu. Very simply, the beginning of my script (written in Vim) looks like this:

#!/bin/bash
set -e
set -u
set -o pipefail

#set variable for samples text file
sample_info=samples.txt

#set variable for fastq files
fq=($(cut -f 3 "$sample_info"))

for fastqc_file in ${fq[@]}
do
        fastqc $fastqc_file
done

and the error I receive is

Skipping '~/Documents/bash_practice/tutorial/UMH_R1a.fastq' which didn't exist, or couldn't be read

The samples.txt is a simple file with 3 tab separated columns containing name, read #, and file path:

UMH R1  ~/Documents/bash_practice/tutorial/UMH_R1a.fastq
UMH R2  ~/Documents/bash_practice/tutorial/UMH_R2a.fastq
UMN R1  ~/Documents/bash_practice/tutorial/UMN_R1a.fastq
UMN R2  ~/Documents/bash_practice/tutorial/UMN_R2a.fastq

I know there is no issue with the file or location because when I run:

fastqc ~/Documents/bash_practice/tutorial/UMH_R1a.fastq

on its own, everything runs fine. So I think the issue must be with how the script is accessing or trying to process my samples.txt file.

I am new to writing scripts, so the answer may be something trivial, and my apologies if it is very obvious.

Thank you in advance, B

software error automation scripting • 577 views
ADD COMMENT
0
Entering edit mode

Arrays are kind of unnecessary here. Given that arrays are almost abusive to the shell, I don't see any merit to this approach. Why not a simple

for fq_file in $(cut -f3 $sample_info)
do
    fastqc $fq_file
done

or even better, use GNU parallel. I can't type the command off the top of my head, but it should be pretty straightforward to create.

ADD REPLY
0
Entering edit mode

Hi, Thanks for the reply. retyped the script exactly as you suggested, and I still receive the same error. When I tell the script to just execute the command on a particular file, it works fine. It seems to be something about it retrieving it from the text file that is an issue. Any suggestions? Thanks,

ADD REPLY
0
Entering edit mode

Can you try changing the ~ to the absolute path to your home directory in the text file? If your home directory is /home/bertb, try:

sed -i 's#~#/home/bertb#' $sample_info

Maybe the shell is seeing ~ as a literal and not as a link to your home directory.

ADD REPLY
0
Entering edit mode

one way to troubleshoot bash scripts is to put an echo in front of your command

echo fastqc $fastqc_file

now it prints the commands, save that into a file and execute those with bash see if it really comes out the way you think it should

 bash myscripts > commands.txt

look at your commands, now execute them:

 bash commands.txt
ADD REPLY

Login before adding your answer.

Traffic: 1171 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6