RAxML multi-threading option in HPC cluster
2
0
Entering edit mode
6.3 years ago
MAPK ★ 2.1k

I am trying to run raxml (https://sco.h-its.org/exelixis/web/software/raxml/hands_on.html) on HPC cluster using multiple cpus using a pbs script below, but for some reason it is only using one cpu and is very slow. I have selected 60 cpus with -T 60, but the process only seems to use a single cpu. Is there anything I am missing here? Thanks for your help in advance.

#!/bin/bash

#PBS -N RaxML
#PBS -l nodes=5:ppn=12
#PBS -l walltime=20:00:00
#PBS -j oe

cd $PBS_O_WORKDIR

raxmlHPC-PTHREADS-SSE3 -T 60 -m PROTGAMMAJTT -p 12345 -# 20 -s aligned_fasta_seq.fas -n T12
raxml phylogenetics • 5.7k views
ADD COMMENT
3
Entering edit mode
6.3 years ago
MAPK ★ 2.1k

So I figured out the answer to this problem. The multithreading in cluster works only when we use raxmlHPC-MPI (MPI version). Here is the command I used (note mpiexec used to execute in multiple cores): mpiexec raxmlHPC-MPI -N $PBS_NP -m PROTGAMMAJTT -p 12345 -s test_mpkk.fas -n T13

Also, -T option is not used in MPI version which should be replace with -N option.

ADD COMMENT
2
Entering edit mode
6.3 years ago
Michael 55k
-# 20 -s aligned_fasta_seq.fas -n T12

I know the -# is an option to raxml to set the number of bootstrap iterations.

Try using -N 20 or -N 100 instead, or quote the # in the command to avoid that this is interpreted as a comment, this doesn't seem to be the problem here but anyway.

Whether parallelized programs will use multiple threads depends on their implementation and the environment.

You are using raxmlHPC- with PTHREADS where most likely you need to use a version built with MPI (raxmlHPC-MPI...)

You requested 5 nodes with 12 cpus each, but that is not the same as having 60 cpus in total on a single computer. raxml-PTHREADS can only use multiple threads on a single computer/node (having 60 CPUs for example) because pthreads cannot communicate across nodes (afaik). In addition, tree searches are run in parallel in the process, so you need to run at least 60 bootstrap iterations to fully profit from 60 cpus, right?

I don't have a PBS cluster to test this at hand, so these are just suggestions from my side.

ADD COMMENT
0
Entering edit mode

Thanks Michael for clarifying.

ADD REPLY

Login before adding your answer.

Traffic: 1805 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6