Question: BWA script on partially executes
0
gravatar for matthew.dambrosio.1
9 months ago by
matthew.dambrosio.10 wrote:

I'm trying to run multiple alignments on my school's pbs cluster. Running the script as an interactive job works perfectly fine, I get all the files .sam files I need in about an hour. When I run it as a non-interactive job it outputs about a 9000 kb .sam file and then finishes the job which is far from complete.

#!/bin/bash
#PBS -l nodes=1:ppn=28,walltime=02:00:00
#PBS -N mapping
#PBS -q short

cd /home/workingdir/

module load shared
module load anaconda/2

i=1

for j in {1..10};
  do
    bwa bwasw -t 3 "mypath/ref.fasta"  'CH2009_split_'$j'_R'$i'.fq.gz' > map/'CH2009_'$j'_R'$i'.sam' &
done

This is the same script I run in the interactive and non-interactive node. I should also say that I've split up one large .fasta file into smaller ones so I can align them simultaneously in the for loop

bash pbs alignment • 224 views
ADD COMMENTlink modified 9 months ago by Philipp Bayer6.6k • written 9 months ago by matthew.dambrosio.10
3
gravatar for Philipp Bayer
9 months ago by
Philipp Bayer6.6k
Australia/Perth/UWA
Philipp Bayer6.6k wrote:

Some PBS systems don't like the & - the jobs get all sent to background, the first job starts writing, but since all jobs are in the background the PBS job finishes.

Have you tried adding a single wait at the end of the last line? Like in this example: https://stackoverflow.com/a/13283453

And since you have so many jobs, it's probably better to learn about PBS job arrays (one job per 'slot' in the array) https://arc-ts.umich.edu/software/torque/job-arrays/

ADD COMMENTlink written 9 months ago by Philipp Bayer6.6k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1818 users visited in the last hour