Question: How to qsub a task.sh (eg. blastn, bowties) that could launch multiple threads
0
gravatar for Mingkun
3.0 years ago by
Mingkun30
China
Mingkun30 wrote:

As a biologist, I am very confused when I am using qsub to submit a task, how to qsub a task that could run in multiple threads, i.e., using -p in bowtie2, -a in blastn. After a exhaustive google search, I have tried -l nodes=1:ppn=8, -l ncpus=8, -pe pe_splots 8, all these parameters do not exist on our SGE server. The only one seems to work is qsub -pe orte 8, however, although 8 cores are assigned, bowtie2 and blastn only run in single threads, anyone could help me on this? 

sge blastn qsub bowtie2 • 1.5k views
ADD COMMENTlink modified 3.0 years ago by genomax51k • written 3.0 years ago by Mingkun30
1
gravatar for Devon Ryan
3.0 years ago by
Devon Ryan81k
Freiburg, Germany
Devon Ryan81k wrote:

On our cluster "-pe smp 20", for example, would specify 20 threads. You should really ask the cluster administrator though, since he/she will be able to tell you exactly the right options.

ADD COMMENTlink written 3.0 years ago by Devon Ryan81k

Thanks, I finally have -pe orte 8 successful running on my terminal now. I have a follow-up question, when I run multiple blastn on my own workstation, they shared the same memory load,i.e.,each of the 10 BLASTN task (using the same reference database NT) took 90% of MEM, whether the different BLASTN tasks could share the memory in the same way or not? or they have to be on the same physical node?

ADD REPLYlink written 3.0 years ago by Mingkun30
1

They'll need to be run on the same node to use the same memory. There's probably a qsub option to put things on the same node, read through man qsub.

ADD REPLYlink written 3.0 years ago by Devon Ryan81k

There are some interesting performance comparisons here: BLAST: Is there a difference between splitting queries and using more threads?

Better performance either by using more threads or by more processes may depend on the dataset you are using.

 

ADD REPLYlink written 3.0 years ago by genomax51k
0
gravatar for genomax
3.0 years ago by
genomax51k
United States
genomax51k wrote:

You need to tell blastn to use multiple threads (it is not clear from your post if you are doing that). Relevant option would be

 -num_threads n

Similarly for bowtie

 -p/--threads <int> number of alignment threads to launch

that number needs to match your qsub command "-pe orte 8".

ADD COMMENTlink written 3.0 years ago by genomax51k

I have done this, now it works.

ADD REPLYlink written 3.0 years ago by Mingkun30
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1575 users visited in the last hour