Getting MPI to work with MEME suite CPU parallelization in OpenSuse

Question

[Joel Wallenius]

Hello!

When I run the MEME suite and specify the number of cores to use, it says I haven't configured the parallelization. Apparently a prerequisite for that is one of the following packages:

• openmpicc
• mpcc
• mpicc

I looked through YaST2 and found only a package called openmpi, rather than openmpicc. Installing this package does not change anything.

How can I get MEME to use parallel computing? It is quite critical for the feasibility of my project.

Thanks a bunch in advance!

Answer 1

[Alex Reynolds]

Here's a makefile I use for building an OpenMPI-ready version of MEME:

SHELL:=/bin/bash

VERSION=4.12.0
MEME_VERSION=meme_${VERSION}
GS=/net/module/sw/ghostscript/9.19/bin/gs
CONVERT=/net/module/sw/ImageMagick/7.0.2-0/bin/convert
MPIDIR=/net/module/sw/openmpi/1.10.2
TEST_FASTA=${PWD}/../data/AG10803-DS12374.fps.bed.head200.fa
RESULTS_DIR=${PWD}/../results
DEST_DIR=${RESULTS_DIR}/${MEME_VERSION}
BIN_DIR=${DEST_DIR}/bin

NP1=1
NP2=2
NP4=4
NP32=32
NP64=64
SLURM_PARTITION=queue1
SLURM_JOB_NAME=parallel_meme_test

all: 

download:
    wget http://meme-suite.org/meme-software/${VERSION}/${MEME_VERSION}.tar.gz

extract:
    tar zxvf ${MEME_VERSION}.tar.gz

build:
    module add openmpi; \
    module add ghostscript; \
    module add ImageMagick; \
    cd ${MEME_VERSION}; \
    ./configure \
        --prefix=${DEST_DIR} \
        --with-url="http://meme.nbcr.net/meme" \
        --with-mpidir=${MPIDIR} \
        --with-gs=${GS} \
        --with-convert=${CONVERT} \
        --enable-opt; \
    ${MAKE}; \
    ${MAKE} install; \
    cd ${PWD}; \

test_local:
    module add openmpi; \
    module add ghostscript; \
    module add ImageMagick; \
    echo "NP=1"; \
    time ${BIN_DIR}/meme -p ${NP1} ${TEST_FASTA} -dna -oc ${RESULTS_DIR}/local_${NP1} -nostatus -time 18000 -maxsize 60000 -mod zoops -nmotifs 10 -minw 6 -maxw 50 -revcomp; \
    echo "NP=2"; \
    time ${BIN_DIR}/meme -p ${NP2} ${TEST_FASTA} -dna -oc ${RESULTS_DIR}/local_${NP2} -nostatus -time 18000 -maxsize 60000 -mod zoops -nmotifs 10 -minw 6 -maxw 50 -revcomp; \
    echo "NP=4"; \
    time ${BIN_DIR}/meme -p ${NP4} ${TEST_FASTA} -dna -oc ${RESULTS_DIR}/local_${NP4} -nostatus -time 18000 -maxsize 60000 -mod zoops -nmotifs 10 -minw 6 -maxw 50 -revcomp; \

test_slurm:
    mkdir -p ${RESULTS_DIR}/slurm_${NP32}; \
    sbatch \
        --partition=${SLURM_PARTITION} \
        --nodes=1 \
        --ntasks-per-node=32 \
        --job-name=${SLURM_JOB_NAME} \
        --output=${RESULTS_DIR}/slurm_${NP32}/stdout.txt \
        --error=${RESULTS_DIR}/slurm_${NP32}/stderr.txt \
        --wrap="module add openmpi; module add ImageMagick; module add ghostscript; time ${BIN_DIR}/meme -p ${NP32} ${TEST_FASTA} -dna -oc ${RESULTS_DIR}/slurm_${NP32} -nostatus -time 18000 -maxsize 60000 -mod zoops -nmotifs 10 -minw 6 -maxw 50 -revcomp;"; \
    mkdir -p ${RESULTS_DIR}/slurm_${NP64}; \
    sbatch \
        --partition=${SLURM_PARTITION} \
        --nodes=2 \
        --ntasks-per-node=32 \
        --job-name=${SLURM_JOB_NAME} \
        --output=${RESULTS_DIR}/slurm_${NP64}/stdout.txt \
        --error=${RESULTS_DIR}/slurm_${NP64}/stderr.txt \
        --wrap="module add openmpi; module add ImageMagick; module add ghostscript; time ${BIN_DIR}/meme -p ${NP64} ${TEST_FASTA} -dna -oc ${RESULTS_DIR}/slurm_${NP64} -nostatus -time 18000 -maxsize 60000 -mod zoops -nmotifs 10 -minw 6 -maxw 50 -revcomp;" \

clean:
    cd ${MEME_VERSION}; \
    ${MAKE} clean; \
    ${MAKE} distclean; \
    cd ${PWD}; \
    rm -rf ${MEME_VERSION}; \
    rm ${MEME_VERSION}.tar.gz; \
    rm -rf ${RESULTS_DIR}/local_* ${RESULTS_DIR}/slurm_*

You'd need to change the values of variables that point to your installation of OpenMPI, Ghostscript and ImageMagick, then make the build target. Hope this helps!

[Joel Wallenius]

Does that file override the default one, or should I concatenate them somehow?

[Alex Reynolds]

I don't really understand your question, but you could replace GS, CONVERT, and MPIDIR with the directories containing Ghostscript, ImageMagick convert, and your OpenMPI installation folder. Replace RESULTS_DIR with a directory where you want MEME installed. Then run make download, make extract, and make build to download and extract the current version of MEME, and build it with OpenMPI support.

[Joel Wallenius]

The MEME readme says to install by:

1. configure --prefix=$HOME/meme --with-url=http://meme-suite.org --enable-build-libxml2 --enable-build-libxslt
2. make
3. make test
4. make install

Are all of those four steps replaced by the steps that you described?

If yes, do I only replace the standard make-file with the one you gave in your first reply?

[Alex Reynolds]

You don't replace anything. You can create a directory, put this makefile in that directory and then follow the steps I describe (edit this makefile, then make the specified targets). This makefile downloads MEME and compiles it with support for OpenMPI.

[Joel Wallenius]

Ah, I understand now. Why do you have such a makefile?

I received help from the OpenSUSE irc chat, apparently meme couldn't on its own find the mpicc/mpirun installation directory. So weird... but it works now.

Thanks a huge bunch though!

[Alex Reynolds]

We use a customized build of MEME for analyzing chromatin data, so having a version that is multiprocessor-capable is useful for us.

[Joel Wallenius]

How customized is it? Could it still be used for standard motif discovery?

[Alex Reynolds]

The makefile I show here contains no such modifications. It makes a stock version of MEME, albeit one that can be run with multiple processors.

[Joel Wallenius]

Alright. I finally got it working by downloading openmpi from the developer's website, installing it in my home folder, and specifying that installation folder while configuring MEME. I will never trust YaST again when it comes to niche or avant-garde software :(

Thank you regardless, Alex!