Question: BWA batch script does not execute
0
gravatar for madkitty
3.2 years ago by
madkitty580
Canada
madkitty580 wrote:

I'm running an alignment with BWA on multiple exomes files on our server. For some reason it does not output anything, and doesn't even seem to be working... any clue why ?!?!

#!/bin/bash
#PBS -l nodes=2:ppn=8
#PBS -l walltime=5:00:00
#PBS -N my_username
cd $PBS_O_WORKDIR

module load bwakit

for i in $(ls *.fastq)
do

bwa mem /mypath/ref/hg38.fa -t 8 /mypath/R1/$i_R1.fastq /mypath/R2/$i_R2.fastq
> /mypath/align/$i\.sam
${i%.fastq}

Done
bwa • 1.4k views
ADD COMMENTlink modified 3.2 years ago by Brice Sarver2.6k • written 3.2 years ago by madkitty580
1

Try executing it on the login node and see what happens...

Hint: done, not Done

ADD REPLYlink modified 3.2 years ago • written 3.2 years ago by Devon Ryan90k

Noooo ... are you trying to get @madkitty banned from this cluster :-)

ADD REPLYlink written 3.2 years ago by genomax68k

Eh, a small pair of input fastq files and lowering the thread count would have made it apparent what the problem was. At least for the clusters I've used there's been a bit of understanding when people try to debug something small on the login node.

ADD REPLYlink written 3.2 years ago by Devon Ryan90k

Per the other thread I linked below @madkitty has 100 files (50 jobs).
I am not a PBS user but at least on LSF every job that is submitted needs to be encapsulated with LSF commands. That is not going to happen with the script above. Perhaps PBS works differently.

ADD REPLYlink modified 3.2 years ago • written 3.2 years ago by genomax68k

My expectation is that this is just a scripting error, which would then be apparent running things on a single server.

ADD REPLYlink written 3.2 years ago by Devon Ryan90k

Re-reading the OP it is not clear if the script is running or it is running and the actual jobs (not sure if the script as written will produce 50 independent jobs) are not producing any output. Part of the problem is likely the file name issue that @Brice has addressed in his answer.

ADD REPLYlink modified 3.2 years ago • written 3.2 years ago by genomax68k

The script is just not doing anything, no files are being creating.

ADD REPLYlink written 3.2 years ago by madkitty580

@madkitty: We had gone over how you would do this in the other thread (C: How to align 100 samples with BWA? ). You are not following the advice there.
You would do module load bwakit and then follow the script in thread mentioned.

ADD REPLYlink modified 3.2 years ago • written 3.2 years ago by genomax68k

Yes I followed the advices there.

ADD REPLYlink written 3.2 years ago by madkitty580
1
gravatar for Brice Sarver
3.2 years ago by
Brice Sarver2.6k
United States
Brice Sarver2.6k wrote:

Four things, some minor:

  1. Make sure the call in bwakit module to execute BWA is actually bwa. Perhaps try on the head node, but don't run it unless you want to get yelled at by your Sys Ad.

  2. Variable 'i' is equal to '[STRING].fastq' Your call, therefore, is malformed. You want to do something like:

    name=$(echo $i | cut -d '.' -f 1); and then let 'name' be the first part of the fastq.

    Edit: I see that you were trying to do this with the % operator, but you need to define this beforehand: name=$(echo $i | cut -d '.' -f 1); or b=${i%.fastq} would store the same value, assuming that it follows the simple structure listed above.

  3. for i in *.fastq; do ... is all you need in this case.

  4. You are using 8 cores, but you are reserving two nodes with 8 processors per node. Seems like nodes=1:ppn=8 would suffice.

  5. [Devon already commented on the 'Done' issue above.]

ADD COMMENTlink modified 3.2 years ago • written 3.2 years ago by Brice Sarver2.6k

Thank you, will give it a try with your advices.

ADD REPLYlink written 3.2 years ago by madkitty580
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1046 users visited in the last hour