Question: BWA script on partially executes
0
gravatar for matthew.dambrosio.1
7 days ago by
matthew.dambrosio.10 wrote:

I'm trying to run multiple alignments on my school's pbs cluster. Running the script as an interactive job works perfectly fine, I get all the files .sam files I need in about an hour. When I run it as a non-interactive job it outputs about a 9000 kb .sam file and then finishes the job which is far from complete.

#!/bin/bash
#PBS -l nodes=1:ppn=28,walltime=02:00:00
#PBS -N mapping
#PBS -q short

cd /home/workingdir/

module load shared
module load anaconda/2

i=1

for j in {1..10};
  do
    bwa bwasw -t 3 "mypath/ref.fasta"  'CH2009_split_'$j'_R'$i'.fq.gz' > map/'CH2009_'$j'_R'$i'.sam' &
done

This is the same script I run in the interactive and non-interactive node. I should also say that I've split up one large .fasta file into smaller ones so I can align them simultaneously in the for loop

bash pbs alignment • 71 views
ADD COMMENTlink modified 7 days ago by Philipp Bayer6.1k • written 7 days ago by matthew.dambrosio.10
3
gravatar for Philipp Bayer
7 days ago by
Philipp Bayer6.1k
Australia/Perth/UWA
Philipp Bayer6.1k wrote:

Some PBS systems don't like the & - the jobs get all sent to background, the first job starts writing, but since all jobs are in the background the PBS job finishes.

Have you tried adding a single wait at the end of the last line? Like in this example: https://stackoverflow.com/a/13283453

And since you have so many jobs, it's probably better to learn about PBS job arrays (one job per 'slot' in the array) https://arc-ts.umich.edu/software/torque/job-arrays/

ADD COMMENTlink written 7 days ago by Philipp Bayer6.1k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1914 users visited in the last hour