How to qsub a task.sh (eg. blastn, bowties) that could launch multiple threads

Question

[Mingkun]

As a biologist, I am very confused when I am using qsub to submit a task, how to qsub a task that could run in multiple threads, i.e., using -p in bowtie2, -a in blastn. After a exhaustive google search, I have tried -l nodes=1:ppn=8, -l ncpus=8, -pe pe_splots 8, all these parameters do not exist on our SGE server. The only one seems to work is qsub -pe orte 8, however, although 8 cores are assigned, bowtie2 and blastn only run in single threads, anyone could help me on this?

Answer 1

[Devon Ryan]

On our cluster "-pe smp 20", for example, would specify 20 threads. You should really ask the cluster administrator though, since he/she will be able to tell you exactly the right options.

[Mingkun]

Thanks, I finally have -pe orte 8 successful running on my terminal now. I have a follow-up question, when I run multiple blastn on my own workstation, they shared the same memory load,i.e.,each of the 10 BLASTN task (using the same reference database NT) took 90% of MEM, whether the different BLASTN tasks could share the memory in the same way or not? or they have to be on the same physical node?

[Devon Ryan]

They'll need to be run on the same node to use the same memory. There's probably a qsub option to put things on the same node, read through man qsub.

[GenoMax]

There are some interesting performance comparisons here: BLAST: Is there a difference between splitting queries and using more threads?

Better performance either by using more threads or by more processes may depend on the dataset you are using.

Answer 2

[GenoMax]

You need to tell blastn to use multiple threads (it is not clear from your post if you are doing that). Relevant option would be

-num_threads n

Similarly for bowtie

-p/--threads <int> number of alignment threads to launch

that number needs to match your qsub command -pe orte 8.

[Mingkun]

I have done this, now it works.