Conda install package - different internals

Question

[lihe.liu]

Dear Biostars community,

I am using Conda to install some R packages on my Mac. (e.g. "gage")

I run "conda install -n myr -c bioconda bioconductor-gage", and It gives me an error like this:

"Error: package or namespace load failed for ‘gage’: package ‘graph’ was installed by an R version with different internals; it needs to be reinstalled for use with this R version."

This happened a few times with installations of other packages.

Every time it happens I create a new env and start over again (sorry it's stupid...I was just testing to figure out what was the problem) and I will get the same error with other packages (seems like pretty random?)

Can anybody help me out? Appreciate your help!

Best, Lihe

[Ram]

If you're using conda to manage R, make sure you don't use install.packages to manage R packages. Use conda install instead. In your case, try conda update -n myr r-graph and then install gage

[lihe.liu]

Thanks you so much! But i actually did not use install.packages() to manage that. I tried the way you suggested and i got the same error....also i am getting errors like " Error: package or namespace load failed for ‘clusterProfiler’: object ‘simplify’ is not exported by 'namespace:IRanges'". Thanks for your patience.. Sorry i am really new to conda...

[Ram]

You might have to reinstall a bunch of packages. Maybe try reinstalling/updating clusterProfiler and see if that helps.

[joe]

In general, if an R package exists in conda you should install via conda, however, if you call R from the conda env and then use install.packages(), the library path will be in the conda environment and everything should work ok, example below. A caveat is that all the correct permissions need to be in place for R to access existing packages.

Just to make sure, you're trying to install R packages into a conda env with R installed? And the R that is installed hasn't been updated?

Also, at the moment it seems like R3.5.1 is the most "stable" https://anaconda.org/r/r/files

Your R packages will be install to ~/anaconda3/envs/r35p27/lib/R/library

(base) joe:~$ conda activate r35p27
(r35p27) joe:~$ R

R version 3.5.1 (2018-07-02) -- "Feather Spray"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-conda_cos6-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> install.packages("ISLR")
--- Please select a CRAN mirror for use in this session ---
Secure CRAN mirrors

 1: 0-Cloud [https]                   2: Algeria [https]
 3: Australia (Canberra) [https]      4: Australia (Melbourne 1) [https]
 5: Australia (Melbourne 2) [https]   6: Australia (Perth) [https]
 7: Austria [https]                   8: Belgium (Ghent) [https]
 9: Brazil (PR) [https]              10: Brazil (RJ) [https]
11: Brazil (SP 1) [https]            12: Brazil (SP 2) [https]
13: Bulgaria [https]                 14: Chile [https]
15: China (Hong Kong) [https]        16: China (Guangzhou) [https]
17: China (Lanzhou) [https]          18: China (Shanghai) [https]
19: Colombia (Cali) [https]          20: Czech Republic [https]
21: Denmark [https]                  22: Ecuador (Cuenca) [https]
23: Ecuador (Quito) [https]          24: Estonia [https]
25: France (Lyon 1) [https]          26: France (Lyon 2) [https]
27: France (Marseille) [https]       28: France (Montpellier) [https]
29: France (Paris 2) [https]         30: Germany (Erlangen) [https]
31: Germany (Göttingen) [https]      32: Germany (Münster) [https]
33: Germany (Regensburg) [https]     34: Greece [https]
35: Hungary [https]                  36: Iceland [https]
37: Indonesia (Jakarta) [https]      38: Ireland [https]
39: Italy (Padua) [https]            40: Japan (Tokyo) [https]
41: Japan (Yonezawa) [https]         42: Korea (Busan) [https]
43: Korea (Gyeongsan-si) [https]     44: Korea (Seoul 1) [https]
45: Korea (Ulsan) [https]            46: Malaysia [https]
47: Mexico (Mexico City) [https]     48: Norway [https]
49: Philippines [https]              50: Serbia [https]
51: Spain (A Coruña) [https]         52: Spain (Madrid) [https]
53: Sweden [https]                   54: Switzerland [https]
55: Turkey (Denizli) [https]         56: Turkey (Mersin) [https]
57: UK (Bristol) [https]             58: UK (London 1) [https]
59: USA (CA 1) [https]               60: USA (IA) [https]
61: USA (KS) [https]                 62: USA (MI 1) [https]
63: USA (MI 2) [https]               64: USA (OR) [https]
65: USA (TN) [https]                 66: USA (TX 1) [https]
67: Uruguay [https]                  68: (other mirrors)


Selection: 1
trying URL 'https://cloud.r-project.org/src/contrib/ISLR_1.2.tar.gz'
Content type 'application/x-gzip' length 1757568 bytes (1.7 MB)
==================================================
downloaded 1.7 MB

* installing *source* package ‘ISLR’ ...
** package ‘ISLR’ successfully unpacked and MD5 sums checked
** data
*** moving datasets to lazyload DB
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (ISLR)

The downloaded source packages are in
    ‘/tmp/RtmpXegJcr/downloaded_packages’
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
> library(ISLR)

[Ram]

Are you sure install.packages for all packages will work fine? For instance, can you show a similar log for tidyverse?

[joe]

tidyverse is on conda, so I would use conda in this case, but out of curiousity I installed the way I describe above and it worked fine. The log is way too long to post everything because of installing dependencies and compiling, but here is the top and bottom of the log...I'll also note that I've had problems doing this with packages having openssl as a dependency.

(base) joe@ip-nnn:~$ conda activate r35p36
(r35p36) joe@ip-nnn:~$ cd ~/anaconda3/envs/r35p36/lib/R/library
(r35p36) joe@ip-nnn:~/anaconda3/envs/r35p36/lib/R/library$ ls -l
.
.
.
(r35p36) joe@ip-nnn:~/anaconda3/envs/r35p36/lib/R/library$ R

R version 3.5.1 (2018-07-02) -- "Feather Spray"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-conda_cos6-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> install.packages("tidyverse")
--- Please select a CRAN mirror for use in this session ---
Secure CRAN mirrors

 1: 0-Cloud [https]                   2: Algeria [https]

.
.
Selection: 1
also installing the dependencies ‘colorspace’, ‘ps’, ‘highr’, ‘markdown’, ‘lattice’, ‘plyr’, ‘Matrix’, ‘labeling’, ‘munsell’, ‘RColorBrewer’, ‘zeallot’, ‘rematch’, ‘prettyunits’, ‘processx’, ‘knitr’, ‘yaml’, ‘htmltools’, ‘evaluate’, ‘tinytex’, ‘xfun’, ‘utf8’, ‘backports’, ‘generics’, ‘nlme’, ‘reshape2’, ‘assertthat’, ‘pkgconfig’, ‘tidyselect’, ‘BH’, ‘plogr’, ‘DBI’, ‘ellipsis’, ‘gtable’, ‘lazyeval’, ‘mgcv’, ‘scales’, ‘viridisLite’, ‘withr’, ‘vctrs’, ‘clipr’, ‘cellranger’, ‘progress’, ‘callr’, ‘fs’, ‘rmarkdown’, ‘whisker’, ‘selectr’, ‘fansi’, ‘pillar’, ‘broom’, ‘cli’, ‘crayon’, ‘dplyr’, ‘dbplyr’, ‘forcats’, ‘ggplot2’, ‘haven’, ‘hms’, ‘lubridate’, ‘modelr’, ‘purrr’, ‘readr’, ‘readxl’, ‘reprex’, ‘rlang’, ‘rstudioapi’, ‘rvest’, ‘tibble’, ‘tidyr’

.
.
.

* installing *source* package ‘tidyverse’ ...
** package ‘tidyverse’ successfully unpacked and MD5 sums checked
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (tidyverse)

The downloaded source packages are in
    ‘/tmp/RtmpyyuK2w/downloaded_packages’
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
> library(tidyverse)
── Attaching packages ─────────────────────────────────────── tidyverse 1.2.1 ──
✔ ggplot2 3.2.0     ✔ purrr   0.3.2
✔ tibble  2.1.3     ✔ dplyr   0.8.3
✔ tidyr   0.8.3     ✔ stringr 1.3.1
✔ readr   1.3.1     ✔ forcats 0.4.0
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
> q()
Save workspace image? [y/n/c]: n
(r35p36) joe@ip-nnn:~/anaconda3/envs/r35p36/lib/R/library$ lrt
.
.
.
drwxrwxr-x  7 joe      users 6144 Aug  6 05:43 tidyverse

[Ram]

Ah, this never worked on my MacBook Pro, probably because I had a messy base environment.

[joe]

Clean conda environments are crucial. Also I had some issues with R3.6 which is why I'm still on R3.5

[lihe.liu]

Thank you so much!

So, I guess I have some hints, I might possibly messed my .bash_profile file.(because i uninstalled Conda and reinstalled it a few times). Now, the situation is that, when I call R within my envs, it is atucally calling the R from the my laptop (or conda base? not sure). It should be calling the R in the specific env i guess.

I wonder what the .bash_profile look like prior to any conda installation and env creation. (Seems that every (re)installation would append some configurations to this file.).

Thank you so much! Appreciate your help!

[Ram]

Your default bash profile should not affect things when you conda activate myenv. Just so you're sure, run both which R as well as which -a R and ensure you're executing the right R. Better, just use the absolute path to R and try installing to see if it works OK.

None of this should matter when you're using conda install -c conda-forge r-package_name though.

[lihe.liu]

Yes, Thank you! Yea I see, the version is not the problem. I have my local R and the R in my env, and I can see different version number when I call them correspondingly. So, all good!

The situation now is that when I install the package (e.g. 'gage') I need there are dependencies (e.g. ‘graph') that are installed by an R version with different internals (as the error indicated). Then I tried to fix this, first, using conda update -n myr bioconductor-gosemsim in this env, and then re-install 'gage'. Unfortunately, this did not work. Then, I tried to install this package within R in my specific environment using install.packages(). and this failed too. This ended up with a warning message "Warning message: package ‘GOSemSim’ is not available (for R version 3.6.0)". Anyways, made no difference to the original issue.....

I have no idea why the dependencies (for example here 'graph') are installed by another R because it should come with the master package (for example here "gage") and be installed at the same time and using same R (the R in my env).

Some advice?

Thank you so much!!!

[Ram]

Do you happen to have a R_LIBS settings in an environment file (~/.Renviron perhaps)? That could interfere with the order of library locations in .libPaths(). But then, if you're facing this error with conda install, it has nothing to do with R settings.

Can you give me the list of packages to install so I can try and replicate this error on my side?

[lihe.liu]

I will try to find that. And also I think conda install works well.

For now I am struggling with some errors:

For package "clusterProfiler":

1. Error: package or namespace load failed for ‘DOSE’: package ‘GOSemSim’ was installed by an R version with different internals; it needs to be reinstalled for use with this R version. Failed with error:  ‘package ‘DOSE’ could not be loaded’
2.Error: package or namespace load failed for ‘clusterProfiler’: object ‘simplify’ is not exported by 'namespace:IRanges'".

For package "gage":

1. Error: package or namespace load failed for ‘gage’: package ‘graph’ was installed by an R version with different internals; it needs to be reinstalled for use with this R version."

Appreciate your help so much!!

[Ram]

If you're using install.packages, please add type='source' for each package as you go down the dependency rabbit hole.

For example,

## dependencies that popped up
install.packages('GoSemSim', type = 'source')
install.packages('graph', type = 'source')

## and now try installing again
install.packages('clusterProfiler', type = 'source')
install.packages('gage', type = 'source')

[lihe.liu]

Installation somehow broke as shown. type = 'source' added but it won't proceed.

https://ibb.co/wynZSRn

Something wrong?

[lihe.liu]

May I ask what is gonna print out if you do "echo $R_HOME". Do we have to specify R_HOME in .bash_profile on our Macs? My friend who is using Conda has the R path (same as which R in that env) printed out if he do "echo $R_HOME". Is this potentially a problem? Thank you!

[Ram]

I've never had to use R_HOME. What does that variable do and why are you using it?

[lihe.liu]

I never used it neither (And I don't think it is necessary on Mac, perhaps?).

We were doing troubleshooting and we just noticed this is a difference and potentially could be an issue......

[Ram]

It seems to be set by R as it starts up. I guess it gets unset when R is exited.

$ echo $R_HOME

$ R --vanilla

R version 3.6.1 (2019-07-05) -- "Action of the Toes"
..
..
..

> R.home()
[1] "/Library/Frameworks/R.framework/Resources"
> system('echo $R_HOME')
/Library/Frameworks/R.framework/Resources
> q()

$ echo $R_HOME

$

However, it does get set on my HPC when I use module load R (setenv R_HOME is part of the module file, and I'm guessing it was placed there for some purpose). If you're just using your local computer, unset the R_HOME variable and see if that changes anything.