Entering edit mode
5.2 years ago
Karma
▴
310
My plan to is to make automated pipeline for processing exome data to derive the vcf file. I coded the alignment step in the following way
#!usr/bin/bash
reference="/media/ngs/ref/Homo_sapiens_assembly19/Homo_sapiens_assembly19.fasta"
outputDir=$1
sampleID=$2
trimmedR1=$3
trimmedR2=$4
exec &> $outputDir/$sampleID.StdOp.Err.txt; #Redirect the stdout and stderror to a text file
echo "Starting Run ==" `date +%d/%m/%Y\ %H:%M:%S` 2>> $outputDir/$sampleID.StdOp.Err.txt
# saving the readGroup to the variable
readGroup="@RG\tID:${sampleID}\tSM:${sampleID}\tLB:${sampleID}\tPL:Illumina"
echo "BWA mem runtime start" `date +%d/%m/%Y\ %H:%M:%S` 2>> $outputDir/$sampleID.StdOp.Err.txt
# bwa mem algorithm is used for alignment
bwa mem -M -t 16 -R "$readGroup" $reference -o $outputDir/$sampleID/${sampleID}_raw.sam $trimmedR1 $trimmedR2 2>> $outputDir/$sampleID.StdOp.Err.txt
echo "BWA mem runtime end" `date +%d/%m/%Y\ %H:%M:%S` 2>> $outputDir/$sampleID.StdOp.Err.txt
and I ran the code as follows:
sh exome_pipeline.sh outputDir AB132 AB132_R1.fq.gz AB132_R2.fq.gz
It gives me an error
[main_mem] fail to open file '/AB132 /AB132_raw.sam' : No such file or directory
How can fix this error?
using plain bash ? you'd better learn how to manage a workflow with nextflow / snakemake...
Agreed. If you are getting started with writing pipelines it is a bit of an extra effort to work yourself into a workflow manager but the effort is worth it. You also would need to invest effort into writing the bash script which should have some basic testing and exiting functions in case things go wrong so that you can easily debug. Use a workflow manger.
Is it not possible to create pipeline with plain bash scripting?
I built a bash-based pipeline for exome sequencing a couple of years ago. From your script above, I gather that your shell experience is quite basic and nowhere close enough to be able to build a robust pipeline that accounts for various edge cases. Please invest time in learning a workflow manager, because you're going to need to invest effort in learning something anyway.
yes it is, but as Pierre Lindenbaum and ATpoint pointed out, it's more future-proof if you go for one of the pipeline manager tools
looks like it does not get the
$outputDir
correctly. and as such tries to write to the root folder / to which you might not have accesscan you post all output from your script (and perhaps add an
echo $outputDir
in it)I cross checked
outputDir
is available as directory. I just manually created a directory 'AB132
' insideoutputDir
and executed the script again. I don't know it is working!!!. The directories arenot in the root folder
if it is related to any permission issues.That is not what lieven.sterck is saying. Check the error message.
fail to open file '/AB132 /AB132_raw.sam'
First,
/AB132
is in fact referring to a root directory. So probablyoutputDir
is wrong, so not correctly pared. Also there is a whitespace in that path which is probably messing up things.If that error is verbatim, you have an unescaped space character after the first
/AB123
. Unix will not play nicely with this.I managed to run this by running
and executed the scrpit
sh exome_pipeline.sh
Why are you using
sudo
here?You've given that script admin privileges and that is not a safe thing to do.You may have treated the symptom of your script failing, but not the cause.
No, OP has changed permissions using a
sudo
command, which is meaningless in this context if OP owns the script file.OP has treated neither. OP is not running
sudo sh exome_pipeline.sh
, justsudo chmod
followed bysh exome_pipeline.sh
.