BWA batch script does not execute
1
0
Entering edit mode
5.0 years ago
madkitty ▴ 620

I'm running an alignment with BWA on multiple exomes files on our server. For some reason it does not output anything, and doesn't even seem to be working... any clue why ?!?!

#!/bin/bash
#PBS -l nodes=2:ppn=8
#PBS -l walltime=5:00:00
#PBS -N my_username
cd $PBS_O_WORKDIR

module load bwakit

for i in $(ls *.fastq)
do

bwa mem /mypath/ref/hg38.fa -t 8 /mypath/R1/$i_R1.fastq /mypath/R2/$i_R2.fastq
> /mypath/align/$i\.sam
${i%.fastq}

Done
bwa • 2.2k views
ADD COMMENT
1
Entering edit mode

Try executing it on the login node and see what happens...

Hint: done, not Done

ADD REPLY
0
Entering edit mode

Noooo ... are you trying to get @madkitty banned from this cluster :-)

ADD REPLY
0
Entering edit mode

Eh, a small pair of input fastq files and lowering the thread count would have made it apparent what the problem was. At least for the clusters I've used there's been a bit of understanding when people try to debug something small on the login node.

ADD REPLY
0
Entering edit mode

Per the other thread I linked below @madkitty has 100 files (50 jobs).
I am not a PBS user but at least on LSF every job that is submitted needs to be encapsulated with LSF commands. That is not going to happen with the script above. Perhaps PBS works differently.

ADD REPLY
0
Entering edit mode

My expectation is that this is just a scripting error, which would then be apparent running things on a single server.

ADD REPLY
0
Entering edit mode

Re-reading the OP it is not clear if the script is running or it is running and the actual jobs (not sure if the script as written will produce 50 independent jobs) are not producing any output. Part of the problem is likely the file name issue that @Brice has addressed in his answer.

ADD REPLY
0
Entering edit mode

The script is just not doing anything, no files are being creating.

ADD REPLY
0
Entering edit mode

@madkitty: We had gone over how you would do this in the other thread (C: How to align 100 samples with BWA? ). You are not following the advice there.
You would do module load bwakit and then follow the script in thread mentioned.

ADD REPLY
0
Entering edit mode

Yes I followed the advices there.

ADD REPLY
1
Entering edit mode
5.0 years ago
Brice Sarver ★ 3.6k

Four things, some minor:

  1. Make sure the call in bwakit module to execute BWA is actually bwa. Perhaps try on the head node, but don't run it unless you want to get yelled at by your Sys Ad.

  2. Variable 'i' is equal to '[STRING].fastq' Your call, therefore, is malformed. You want to do something like:

    name=$(echo $i | cut -d '.' -f 1); and then let 'name' be the first part of the fastq.

    Edit: I see that you were trying to do this with the % operator, but you need to define this beforehand: name=$(echo $i | cut -d '.' -f 1); or b=${i%.fastq} would store the same value, assuming that it follows the simple structure listed above.

  3. for i in *.fastq; do ... is all you need in this case.

  4. You are using 8 cores, but you are reserving two nodes with 8 processors per node. Seems like nodes=1:ppn=8 would suffice.

  5. [Devon already commented on the 'Done' issue above.]

ADD COMMENT
0
Entering edit mode

Thank you, will give it a try with your advices.

ADD REPLY

Login before adding your answer.

Traffic: 1987 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6