Question: BWA batch script does not execute
0
gravatar for madkitty
4.2 years ago by
madkitty600
Canada
madkitty600 wrote:

I'm running an alignment with BWA on multiple exomes files on our server. For some reason it does not output anything, and doesn't even seem to be working... any clue why ?!?!

#!/bin/bash
#PBS -l nodes=2:ppn=8
#PBS -l walltime=5:00:00
#PBS -N my_username
cd $PBS_O_WORKDIR

module load bwakit

for i in $(ls *.fastq)
do

bwa mem /mypath/ref/hg38.fa -t 8 /mypath/R1/$i_R1.fastq /mypath/R2/$i_R2.fastq
> /mypath/align/$i\.sam
${i%.fastq}

Done
bwa • 1.9k views
ADD COMMENTlink modified 4.2 years ago by Brice Sarver3.5k • written 4.2 years ago by madkitty600
1

Try executing it on the login node and see what happens...

Hint: done, not Done

ADD REPLYlink modified 4.2 years ago • written 4.2 years ago by Devon Ryan95k

Noooo ... are you trying to get @madkitty banned from this cluster :-)

ADD REPLYlink written 4.2 years ago by genomax85k

Eh, a small pair of input fastq files and lowering the thread count would have made it apparent what the problem was. At least for the clusters I've used there's been a bit of understanding when people try to debug something small on the login node.

ADD REPLYlink written 4.2 years ago by Devon Ryan95k

Per the other thread I linked below @madkitty has 100 files (50 jobs).
I am not a PBS user but at least on LSF every job that is submitted needs to be encapsulated with LSF commands. That is not going to happen with the script above. Perhaps PBS works differently.

ADD REPLYlink modified 4.2 years ago • written 4.2 years ago by genomax85k

My expectation is that this is just a scripting error, which would then be apparent running things on a single server.

ADD REPLYlink written 4.2 years ago by Devon Ryan95k

Re-reading the OP it is not clear if the script is running or it is running and the actual jobs (not sure if the script as written will produce 50 independent jobs) are not producing any output. Part of the problem is likely the file name issue that @Brice has addressed in his answer.

ADD REPLYlink modified 4.2 years ago • written 4.2 years ago by genomax85k

The script is just not doing anything, no files are being creating.

ADD REPLYlink written 4.2 years ago by madkitty600

@madkitty: We had gone over how you would do this in the other thread (C: How to align 100 samples with BWA? ). You are not following the advice there.
You would do module load bwakit and then follow the script in thread mentioned.

ADD REPLYlink modified 4.2 years ago • written 4.2 years ago by genomax85k

Yes I followed the advices there.

ADD REPLYlink written 4.2 years ago by madkitty600
1
gravatar for Brice Sarver
4.2 years ago by
Brice Sarver3.5k
United States
Brice Sarver3.5k wrote:

Four things, some minor:

  1. Make sure the call in bwakit module to execute BWA is actually bwa. Perhaps try on the head node, but don't run it unless you want to get yelled at by your Sys Ad.

  2. Variable 'i' is equal to '[STRING].fastq' Your call, therefore, is malformed. You want to do something like:

    name=$(echo $i | cut -d '.' -f 1); and then let 'name' be the first part of the fastq.

    Edit: I see that you were trying to do this with the % operator, but you need to define this beforehand: name=$(echo $i | cut -d '.' -f 1); or b=${i%.fastq} would store the same value, assuming that it follows the simple structure listed above.

  3. for i in *.fastq; do ... is all you need in this case.

  4. You are using 8 cores, but you are reserving two nodes with 8 processors per node. Seems like nodes=1:ppn=8 would suffice.

  5. [Devon already commented on the 'Done' issue above.]

ADD COMMENTlink modified 4.2 years ago • written 4.2 years ago by Brice Sarver3.5k

Thank you, will give it a try with your advices.

ADD REPLYlink written 4.2 years ago by madkitty600
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1380 users visited in the last hour