Best way to install R packages on an HPC?

Question

[ChumBucket2024]

Hi everyone,

I have been trying to practice single cell analysis using R, and I realized that most of my time goes to installing packages using BiocManager::install or install.packages. A single package can take as long as 45 minutes.

I am using an HPC to do this. When I've tried using mamba install to install packages, I always run into the error:

EnvironmentNotWritableError: The current user does not have write permissions to the target environment.

Does anyone else know how to quickly install R packages on an HPC, or a way around this "not writable" error?

Thank you.

[QX]

I have no idea if they have some environment manager like conda. but I usually make a specific path for R packages for each project,

# setting R-packages path    
lib_path <- paste0(project_dir,'/bin/r_libs')
.libPaths(c(lib_path))

setting this every time you want to install the R packages or import the packages, then it is not violate the admin's R lib.

[GenoMax]

There is some additional detail in https://services.dartmouth.edu/TDClient/1806/Portal/KB/ArticleDet?ID=141373

While using a personal library to install R-packages, as mentioned by others, is likely the solution for this issue, you will likely run into cases where you may need to switch R versions, compile packages etc. Above doc should be helpful with that.

Answer 1

[dariober]

I am using an HPC to do this. When I've tried using mamba install to install packages, I always run into the error:

EnvironmentNotWritableError: The current user does not have write permissions to the target environment.

My guess is that you are using a mamba installation setup by the HPC admin and that installation is configured to write to a directory where regular users do not have writing permission. Have a look at the output of which mamba and mamba info to see if that could be the case. If so, just install your own conda and work with that.

(As an aside, I lost count of how many times I recommended people to use mamba instead of conda. Now it turns out that conda has incorporated the improvements of mamba and so we should probably get back to prefer conda. In fact, newer versions of snakemake support conda but not mamba to run rules in a separate conda environment.)

[cfos4698]

FWIW, I've recently still been running into environment solving issues with conda (even with the incorporated libmamba changes) that mamba breezed through

Answer 2

[Charles Plessy]

At my workplace I created a Singularity image that has most of the C/C++ libraries installed, and R's library path pointing to an image-specific subdirectory in the user's home directory. https://github.com/oist/BioinfoUgrp/tree/master/RStudio