error when running NGS tools in bash script

Question

[bertb]

Hello,

I am trying to write a script to automate my RNAseq pipeline in a Bash shell on Ubuntu. Very simply, the beginning of my script (written in Vim) looks like this:

#!/bin/bash
set -e
set -u
set -o pipefail

#set variable for samples text file
sample_info=samples.txt

#set variable for fastq files
fq=($(cut -f 3 "$sample_info"))

for fastqc_file in ${fq[@]}
do
        fastqc $fastqc_file
done

and the error I receive is

Skipping '~/Documents/bash_practice/tutorial/UMH_R1a.fastq' which didn't exist, or couldn't be read

The samples.txt is a simple file with 3 tab separated columns containing name, read #, and file path:

UMH R1  ~/Documents/bash_practice/tutorial/UMH_R1a.fastq
UMH R2  ~/Documents/bash_practice/tutorial/UMH_R2a.fastq
UMN R1  ~/Documents/bash_practice/tutorial/UMN_R1a.fastq
UMN R2  ~/Documents/bash_practice/tutorial/UMN_R2a.fastq

I know there is no issue with the file or location because when I run:

fastqc ~/Documents/bash_practice/tutorial/UMH_R1a.fastq

on its own, everything runs fine. So I think the issue must be with how the script is accessing or trying to process my samples.txt file.

I am new to writing scripts, so the answer may be something trivial, and my apologies if it is very obvious.

Thank you in advance, B

[Ram]

Arrays are kind of unnecessary here. Given that arrays are almost abusive to the shell, I don't see any merit to this approach. Why not a simple

for fq_file in $(cut -f3 $sample_info)
do
    fastqc $fq_file
done

or even better, use GNU parallel. I can't type the command off the top of my head, but it should be pretty straightforward to create.

[bertb]

Hi, Thanks for the reply. retyped the script exactly as you suggested, and I still receive the same error. When I tell the script to just execute the command on a particular file, it works fine. It seems to be something about it retrieving it from the text file that is an issue. Any suggestions? Thanks,

[Ram]

Can you try changing the ~ to the absolute path to your home directory in the text file? If your home directory is /home/bertb, try:

sed -i 's#~#/home/bertb#' $sample_info

Maybe the shell is seeing ~ as a literal and not as a link to your home directory.

[Istvan Albert]

one way to troubleshoot bash scripts is to put an echo in front of your command

echo fastqc $fastqc_file

now it prints the commands, save that into a file and execute those with bash see if it really comes out the way you think it should

 bash myscripts > commands.txt

look at your commands, now execute them:

 bash commands.txt