BWA script on partially executes
1
0
Entering edit mode
22 months ago

I'm trying to run multiple alignments on my school's pbs cluster. Running the script as an interactive job works perfectly fine, I get all the files .sam files I need in about an hour. When I run it as a non-interactive job it outputs about a 9000 kb .sam file and then finishes the job which is far from complete.

#!/bin/bash
#PBS -l nodes=1:ppn=28,walltime=02:00:00
#PBS -N mapping
#PBS -q short

cd /home/workingdir/

module load shared
module load anaconda/2

i=1

for j in {1..10};
  do
    bwa bwasw -t 3 "mypath/ref.fasta"  'CH2009_split_'$j'_R'$i'.fq.gz' > map/'CH2009_'$j'_R'$i'.sam' &
done

This is the same script I run in the interactive and non-interactive node. I should also say that I've split up one large .fasta file into smaller ones so I can align them simultaneously in the for loop

alignment bash pbs • 388 views
ADD COMMENT
3
Entering edit mode
22 months ago

Some PBS systems don't like the & - the jobs get all sent to background, the first job starts writing, but since all jobs are in the background the PBS job finishes.

Have you tried adding a single wait at the end of the last line? Like in this example: https://stackoverflow.com/a/13283453

And since you have so many jobs, it's probably better to learn about PBS job arrays (one job per 'slot' in the array) https://arc-ts.umich.edu/software/torque/job-arrays/

ADD COMMENT

Login before adding your answer.

Traffic: 2430 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6