Question: Mapping hundreds of single reads to reference genome with bwa mem using shell script
gravatar for t86dan
3.3 years ago by
t86dan30 wrote:

Hi guys,

I have 100+ reads ready to be mapped to the reference genome using the bwa mem algorithm. The read files are in .fastq.gz format (ie. Blank.fastq.gz, TR8.fastq.gz, JE997.fastq.gz, IF898.fastq.gz, U1.fastq.gz, U2.fastq.gz, U3.fastq.gz,... etc.). I am very new to programing and tried running a shell script that lists all the samples and then calls another shell script as follows:


./BwaMapping Blank
./BwaMapping TR8
./BwaMapping JE997
./BwaMapping IF898
./BwaMapping U1
./BwaMapping U2
./BwaMapping U3
./BwaMapping U4

...Etc with all the remaining samples (100+ total)


 # input files


 # map the reads with bwa mem

bwa mem -t 2 -w 600 -R "@RG\tID:${p}\tSM:${p}\tPL:ILLUMINA" ${REFERENCE} ${f1} > ${p}_bwa_mem.sam &

So when I run ./SamplesScript it calls the other script and does the bwa mem command with each sample. The problem is that it starts to process all of the samples at once, running out of memory and obviously crashing. What do I need to change to the script so that it runs the samples in succession one after the other and not all at the same time?

script bwa mapping • 1.6k views
ADD COMMENTlink modified 3.3 years ago by igor9.1k • written 3.3 years ago by t86dan30
gravatar for WouterDeCoster
3.3 years ago by
WouterDeCoster42k wrote:

The easiest would probably be to just use a for loop in your shell, for example

for fastqfile in *.fastq.gz
bwa mem -t 2 -w (etc etc)
ADD COMMENTlink modified 3.3 years ago • written 3.3 years ago by WouterDeCoster42k

It worked, thank you!

ADD REPLYlink written 3.3 years ago by t86dan30
gravatar for igor
3.3 years ago by
United States
igor9.1k wrote:

In your script, you have & at the end of bwa mem command. That sends the process to the background. Remove that and your commands will wait for the previous ones to finish before they start.

ADD COMMENTlink written 3.3 years ago by igor9.1k

Oh I didn't know that was causing the problem! Well I will keep it in mind from now on thanks!

ADD REPLYlink written 3.3 years ago by t86dan30
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 877 users visited in the last hour