Problem with Mamba
1
0
Entering edit mode
11 months ago

I have an issue with Mamba, it seems to run very well on a single node, but when run in a slurm script, it does not recognize the arguments and gives an error message. Below, an example from a Snakemake run. Did it happen to anyone else? Any suggestions what can be wrong here?

Building DAG of jobs...
Creating conda environment https://github.com/snakemake/snakemake-wrappers/raw/v2.6.0/bio/bwa/index/environment.yaml...
Downloading and installing remote packages.
CreateCondaEnvironmentException:
Could not create conda environment from /tmp/tmichel_12774347/tmpoorpm5cp.yaml:
Command:
mamba env create --quiet --file "/mnt/shared/scratch/usr/ROH_pipeline/Begonia_full_genome/.snakemake/conda/1e394e56e793f957129208ab3debae09_.yaml" --prefix "/mnt/shared/scratch/usr/ROH_pipeline/Begonia_full_genome/.snakemake/conda/1e394e56e793f957129208ab3debae09_"
Output:
usage: mamba [-h] [--version] [--slow SLOW] [--enable-coverage]
             [--coverage-file COVERAGE_FILE] [--format FORMAT] [--no-color]
             [--tags TAGS]
             [specs ...]
mamba: error: unrecognized arguments: --quiet --file /mnt/shared/scratch/usr/ROH_pipeline/Begonia_full_genome/.snakemake/conda/1e394e56e793f957129208ab3debae09_.yaml --prefix /mnt/shared/scratch/usr/ROH_pipeline/Begonia_full_genome/.snakemake/conda/1e394e56e793f957129208ab3debae09_
snakemake mamba conda • 2.1k views
ADD COMMENT
2
Entering edit mode

i bet there is some script in your .bashrc (or the one you inherited from root) that is putting you in a hostile conda environment when snakemake dispatches to that worker node.

i would try ssh'ing directly to any node and try to run some commands: mamba env create --quiet --name testenv

ADD REPLY
3
Entering edit mode

I'm adding 'hostile conda environment' to my normal lexicon.

ADD REPLY
1
Entering edit mode

Can you check if the conda/mamba executables you use are the same on both the node you ran things successfully on and on the compute node?

Also, your sysadmin is in a much better position to help you than we are.

ADD REPLY
0
Entering edit mode
11 months ago

I have solved my problem just forcing the use of conda with Snakemake. But out of curiosity I have made some test, and found slightly different version of conda in different nodes.

Test of the version of the software:

Single node:

conda -V
conda 23.3.1
mamba -V
conda 23.3.1

slurm script:

conda -V
conda 23.9.0
mamba -V
conda 23.3.1

Try to make a new environment:

mamba env create --quiet --name testenv
EnvironmentFileNotFound: '/mnt/shared/home/usr/environment.yml' file not found

In the .bashrc file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/usr/miniforge3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/usr/miniforge3/etc/profile.d/conda.sh" ]; then
        . "/home/usr/miniforge3/etc/profile.d/conda.sh"
    else
        export PATH="/home/usr/miniforge3/bin:$PATH"
    fi
fi
unset __conda_setup

if [ -f "/home/usr/miniforge3/etc/profile.d/mamba.sh" ]; then
    . "/home/usr/miniforge3/etc/profile.d/mamba.sh"
fi
# <<< conda initialize <<<
ADD COMMENT
1
Entering edit mode

There have been some issues in the combination of conda and mamba with conda version 23.5 (and maybe lower). You just might need to update or reinstall a recent version, of possible. I don’t find the GitHub thread anymore, but I will look tomorrow. IIRC it was at the same time as the introduction of the new libmamba-solver for conda.

ADD REPLY
1
Entering edit mode

Yeah, I ran into an issue with a similar background as well. Time to switch to miniforge?

OP seems to be running into a different sort of problem where their login node and compute node are seeing different versions of conda. Probably something to do with OP's profile files.

ADD REPLY
0
Entering edit mode

I just installed the new miniconda and it worked again. I also thought about miniforge, but I didn’t want to :).

ADD REPLY
0
Entering edit mode

Since your using slurm, I assume your working on an HPC. Somehow your folder structure is a bit weird. Looks like conda is installed directly in /home/usr. Is /home/usr a symlink to your actual home directory? Otherwise it looks to me like miniforge is installed somehow globally for all users. Don‘t you install it manually on your user folder?

ADD REPLY

Login before adding your answer.

Traffic: 980 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6