Tips On Compiling And Using Meme 4.3 With A Sun Grid Engine Computation Cluster

Question

[Alex Reynolds]

Has anyone compiled and used MEME 4.x for use in a parallel computation environment, based upon operation with a Sun Grid Engine (SGE) cluster?

I can compile the suite and its tests pass. However, when I attempt to use the -p n option, to specify n computation nodes, I get several error messages:

/gridware/codine/util/arch: Command not found.
/gridware/codine/util/arch: Command not found.
/gridware/codine/util/arch: Command not found.
/gridware/codine/util/arch: Command not found.
1: Command not found.

We do not have /gridware/codine/util/arch, but we do have /gridengine/sgi/util/arch.

I tried looking around MEME's source code, particularly at meme.c and mp.h, but there are no references to these paths.

I'm wondering if I am missing makefile directives. Here is my ./configure statement:

./configure --prefix=/home/areynolds/proj/meme/meme_4.3.0_build --with-url="http://meme.nbcr.net/meme" --enable-openmp --enable-debug

Is MPI a requirement; are there directives I am missing for MPI?

Thank you for any advice.

EDIT

I was able to successfully build a version of MEME 4.3 that supports OpenMPI.

First, I worked with our sys admin to install OpenMPI 1.4 on each of the SGE nodes (compilation options) and set up an SGE parallel environment called mpi_test:

$ qconf -sp mpi_test
pe_name           mpi_test
slots             120 
user_lists        NONE
xuser_lists       NONE
start_proc_args   /bin/true
stop_proc_args    /bin/true     
allocation_rule   $fill_up
control_slaves    TRUE
job_is_first_task FALSE 
urgency_slots     min

Secondly, I used the following build options for MEME:

$ ./configure --prefix=/home/areynolds/proj/meme/meme_4.3.0_build \ 
--with-url="http://meme.nbcr.net/meme" \
--enable-openmp \
--enable-debug \
--with-mpicc=/opt/openmpi-1.4/bin/mpicc \
--enable-opt

Thirdly, the mpirun and meme_p binaries depend upon shared libraries in the OpenMPI installation. It is necessary to add /opt/openmpi-1.4/lib to your local LD_LIBRARY_PATH environment variable before running qsub.

Finally, I set up a script (called runall.cluster) along the following lines. Runtime options can be adjusted to taste:

#!/bin/bash

#
# runall.cluster
#

#$ -N memeCluster64
#$ -S /bin/bash
#$ -pe mpi_test 64
#$ -v -np=64
#$ -cwd
#$ -o "memeCluster64.out"
#$ -e "memeCluster64.err"
#$ -notify
#$ -V

time /opt/openmpi-1.4/bin/mpirun -np 64 \
   /home/areynolds/proj/meme/meme/bin/meme_p \
   /home/areynolds/proj/meme/data/K562.DS9767.fps.not.in.promoters.fa.top10k.fa \
   -oc /home/areynolds/proj/meme/output/K562.DS9767.fps.not.in.promoters.10k.cluster.64 \
   -maxsize 230000 \
   -dna \
   -minw 8 -maxw 12 -allw \
   -p 64

This script is executed directly with qsub and runs the parallelized MEME binary meme_p with the specified options. In this case, it will run on 64 nodes of our computation cluster:

qsub ./runall.cluster

Hopefully this information will help out others.

[Istvan Albert]

thanks for following up on this, great information

[Khader Shameer]

This is the real spirit of Biostar, thanks for sharing this.

Answer 1

[Timothy Bailey]

I've put a link on the MEME User Forum to this thread.

Answer 2

[Istvan Albert]

This may not be overly useful but it very much sounds like a configuration problem.

Usually there is configure flag that needs to be set to point to the libraries, something like:

--with-mpidir=MPIDIR
--with-mpicc=MPICC

It also appears that the MEME suite does not support Open MPI (as per install notes).

I would also recommend posting on the MEME user forum