Create Conda Environment with YAML File
1
1
Entering edit mode
6 weeks ago
cthangav ▴ 100

Hello, I'm trying to follow these instructions to create a conda environment with this YAML file but the environment keeps getting stuck on solving.

name: p2b1
dependencies:
- python=3.7.5
- scipy=1.5.3
- keras=2.4.3
- h5py=2.10.0
- numpy=1.19.5
- tensorflow-gpu=2.2.0
- pandas=1.2.0
- scikit-learn=0.24.0
- tqdm=4.56.0
- matplotlib=3.3.3
- numba=0.52.0
- astropy=4.2
- patsy=0.5.1
- statsmodels=0.12.1.

It works when I only include numpy in the list of packages and add conda forge and defaults into the YAML file. I am using conda version 23.3.1 Any way to resolve this?

Python • 494 views
ADD COMMENT
1
Entering edit mode

Please also provide the error messages and command line used, otherwise we can't see what is going wrong.

ADD REPLY
1
Entering edit mode

I used this command: conda env create -f python_environment.yml -n P2B1

And there's no error message, just that it gets stuck on solving the environment for hours until my terminal times out.

ADD REPLY
1
Entering edit mode

You can also try using mamba, I find it can often solve environments that base conda struggles with.

EDIT, only just noticed that it was micromamba that was suggested below as an answer, I originally misread that as miniconda.

ADD REPLY
0
Entering edit mode

Which system are you on? I also noticed that base conda is hardly usable anymore on our systems. In contrast, solving your environment with micromamba took 30 seconds.

ADD REPLY
3
Entering edit mode
6 weeks ago
Michael 54k

It worked for me. It may just take a long time because there are a lot of dependencies to resolve. Try micromamba instead for a faster solve. And once you have the environment, export it completely into a yaml file. This should also help to speed the solving up the next time you install it.

Here is the solved environment:

name: p2b1
channels:
- conda-forge
- pkgs/main
dependencies:
- _libgcc_mutex=0.1=conda_forge
- _openmp_mutex=4.5=2_gnu
- _tflow_select=2.1.0=gpu
- absl-py=2.1.0=pyhd8ed1ab_0
- aiohttp=3.7.4.post0=py37h5e8e339_1
- astor=0.8.1=pyh9f0ad1d_0
- astropy=4.2.1=py37h5e8e339_0
- astunparse=1.6.3=pyhd8ed1ab_0
- async-timeout=3.0.1=py_1000
- attrs=23.2.0=pyh71513ae_0
- blinker=1.6.3=pyhd8ed1ab_0
- brotli-python=1.0.9=py37hd23a5d3_7
- c-ares=1.27.0=hd590300_0
- ca-certificates=2024.2.2=hbcca054_0
- cachetools=5.3.3=pyhd8ed1ab_0
- certifi=2024.2.2=pyhd8ed1ab_0
- cffi=1.14.6=py37hc58025e_0
- chardet=4.0.0=py37h89c1867_3
- charset-normalizer=3.3.2=pyhd8ed1ab_0
- click=8.1.3=py37h89c1867_0
- cryptography=39.0.1=py37h9ce1e76_0
- cudatoolkit=10.1.243=h6d9799a_13
- cudnn=7.6.5.32=hc0a50b0_1
- cupti=10.1.168=0
- cycler=0.11.0=pyhd8ed1ab_0
- dbus=1.13.18=hb2f20db_0
- expat=2.6.2=h59595ed_0
- fontconfig=2.14.2=h14ed4e7_0
- freetype=2.12.1=h267a509_2
- gast=0.3.3=py_0
- glib=2.69.1=h4ff587b_1
- google-auth=2.28.2=pyhca7485f_0
- google-auth-oauthlib=0.4.6=pyhd8ed1ab_0
- google-pasta=0.2.0=pyh8c360ce_0
- grpc-cpp=1.48.1=h05bd8bd_1
- grpcio=1.48.1=py37h3fae8ff_1
- gst-plugins-base=1.14.1=h6a678d5_1
- gstreamer=1.14.1=h5eee18b_1
- h5py=2.10.0=nompi_py37ha3df211_106
- hdf5=1.10.6=nompi_h6a2412b_1114
- icu=73.2=h59595ed_0
- idna=3.6=pyhd8ed1ab_0
- importlib-metadata=4.11.4=py37h89c1867_0
- joblib=1.3.2=pyhd8ed1ab_0
- jpeg=9e=h0b41bf4_3
- keras=2.4.3=pyhd8ed1ab_0
- keras-preprocessing=1.1.2=pyhd8ed1ab_0
- keyutils=1.6.1=h166bdaf_0
- kiwisolver=1.4.4=py37h7cecad7_0
- krb5=1.20.1=hf9c8cef_0
- lcms2=2.12=hddcbb42_0
- ld_impl_linux-64=2.40=h41732ed_0
- lerc=3.0=h9c3ff4c_0
- libabseil=20220623.0=cxx17_h05df665_6
- libblas=3.9.0=20_linux64_openblas
- libcblas=3.9.0=20_linux64_openblas
- libclang=14.0.6=default_h7634d5b_1
- libclang13=14.0.6=default_h9986a30_1
- libcups=2.3.3=h36d4200_3
- libcurl=7.87.0=h6312ad2_0
- libdeflate=1.10=h7f98852_0
- libedit=3.1.20191231=he28a2e2_2
- libev=4.33=hd590300_2
- libexpat=2.6.2=h59595ed_0
- libffi=3.3=h58526e2_2
- libgcc-ng=13.2.0=h807b86a_5
- libgfortran-ng=13.2.0=h69a702a_5
- libgfortran5=13.2.0=ha4646dd_5
- libgomp=13.2.0=h807b86a_5
- libiconv=1.17=hd590300_2
- liblapack=3.9.0=20_linux64_openblas
- libllvm10=10.0.1=he513fc3_3
- libllvm14=14.0.6=hcd5def8_4
- libnghttp2=1.51.0=hdcd2b5c_0
- libopenblas=0.3.25=pthreads_h413a1c8_0
- libpng=1.6.43=h2797004_0
- libpq=12.15=h37d81fd_1
- libprotobuf=3.21.8=h6239696_0
- libsqlite=3.45.2=h2797004_0
- libssh2=1.10.0=haa6b8db_3
- libstdcxx-ng=13.2.0=h7e041cc_5
- libtiff=4.3.0=h0fcbabc_4
- libuuid=2.38.1=h0b41bf4_0
- libwebp-base=1.3.2=hd590300_0
- libxcb=1.15=h0b41bf4_0
- libxkbcommon=1.6.0=hd429924_1
- libxml2=2.12.5=h232c23b_0
- libzlib=1.2.13=hd590300_5
- llvmlite=0.35.0=py37h9d7f4d0_1
- markdown=3.5.2=pyhd8ed1ab_0
- markupsafe=2.1.1=py37h540881e_1
- matplotlib=3.3.3=py37h89c1867_0
- matplotlib-base=3.3.3=py37h0c9df89_0
- multidict=6.0.2=py37h540881e_1
- mysql=5.7.20=hf484d3e_1001
- ncurses=6.4=h59595ed_2
- numba=0.52.0=py37hdc94413_0
- numpy=1.19.5=py37h3e96413_3
- oauthlib=3.2.2=pyhd8ed1ab_0
- olefile=0.47=pyhd8ed1ab_0
- openjpeg=2.5.0=h7d73246_0
- openssl=1.1.1w=hd590300_0
- opt_einsum=3.3.0=pyhc1e730c_2
- packaging=23.2=pyhd8ed1ab_0
- pandas=1.2.0=py37hdc94413_1
- patsy=0.5.1=py_0
- pcre=8.45=h9c3ff4c_0
- pillow=8.4.0=py37h0f21c89_0
- pip=24.0=pyhd8ed1ab_0
- protobuf=4.21.8=py37hd23a5d3_0
- pthread-stubs=0.4=h36c2ea0_1001
- pyasn1=0.5.1=pyhd8ed1ab_0
- pyasn1-modules=0.3.0=pyhd8ed1ab_0
- pycparser=2.21=pyhd8ed1ab_0
- pyerfa=2.0.0.1=py37hda87dfa_2
- pyjwt=2.8.0=pyhd8ed1ab_1
- pyopenssl=23.2.0=pyhd8ed1ab_1
- pyparsing=3.1.2=pyhd8ed1ab_0
- pyqt=5.15.4=py37hd23a5d3_0
- pyqt5-sip=12.9.0=py37hd23a5d3_0
- pysocks=1.7.1=py37h89c1867_5
- python=3.7.5=hffdb5ce_0_cpython
- python-dateutil=2.9.0=pyhd8ed1ab_0
- python_abi=3.7=4_cp37m
- pytz=2024.1=pyhd8ed1ab_0
- pyu2f=0.1.5=pyhd8ed1ab_0
- pyyaml=6.0=py37h540881e_4
- qt-main=5.15.2=h110a718_10
- re2=2022.06.01=h27087fc_1
- readline=8.2=h8228510_1
- requests=2.31.0=pyhd8ed1ab_0
- requests-oauthlib=1.4.0=pyhd8ed1ab_0
- rsa=4.9=pyhd8ed1ab_0
- scikit-learn=0.24.0=py37h69acf81_0
- scipy=1.5.3=py37h14a347d_0
- setuptools=59.8.0=py37h89c1867_1
- sip=6.5.1=py37hcd2ae1e_2
- six=1.16.0=pyh6c4a22f_0
- sqlite=3.45.2=h2c6b66d_0
- statsmodels=0.12.1=py37h902c9e0_2
- tensorboard=2.11.2=pyhd8ed1ab_0
- tensorboard-data-server=0.6.0=py37h38fbfac_2
- tensorboard-plugin-wit=1.8.1=pyhd8ed1ab_0
- tensorflow=2.2.0=gpu_py37h1a511ff_0
- tensorflow-base=2.2.0=gpu_py37h8a81be8_0
- tensorflow-estimator=2.6.0=py37hcd2ae1e_0
- tensorflow-gpu=2.2.0=h0d30ee6_0
- termcolor=2.3.0=pyhd8ed1ab_0
- threadpoolctl=3.1.0=pyh8a188c0_0
- tk=8.6.13=noxft_h4845f30_101
- toml=0.10.2=pyhd8ed1ab_0
- tornado=6.2=py37h540881e_0
- tqdm=4.56.0=pyhd8ed1ab_0
- typing-extensions=4.7.1=hd8ed1ab_0
- typing_extensions=4.7.1=pyha770c72_0
- urllib3=2.2.1=pyhd8ed1ab_0
- werkzeug=2.2.3=pyhd8ed1ab_0
- wheel=0.42.0=pyhd8ed1ab_0
- wrapt=1.14.1=py37h540881e_0
- xkeyboard-config=2.41=hd590300_0
- xorg-kbproto=1.0.7=h7f98852_1002
- xorg-libx11=1.8.7=h8ee46fc_0
- xorg-libxau=1.0.11=hd590300_0
- xorg-libxdmcp=1.1.3=h7f98852_0
- xorg-xextproto=7.3.0=h0b41bf4_1003
- xorg-xproto=7.0.31=h7f98852_1007
- xz=5.2.6=h166bdaf_0
- yaml=0.2.5=h7f98852_2
- yarl=1.7.2=py37h540881e_2
- zipp=3.15.0=pyhd8ed1ab_0
- zlib=1.2.13=hd590300_5
- zstd=1.5.5=hfc55251_0
ADD COMMENT
0
Entering edit mode

To tack on this suggestion to make the solving faster, pixi is another faster way to work with conda.

ADD REPLY
0
Entering edit mode

Good to know, which version of conda are you using? And which channels did you use to create the environment? I may be missing a needed channel.

ADD REPLY
0
Entering edit mode
(p2b1) ubuntu@ubuntu-compute:~$ micromamba --version
1.5.3

I have tested this in a Ubuntu 22 LTS VM that is using the BioStar handbook bioinfo setup (insecure connection). Additional channels enabled are bioconda, R, and conda-forge, however there doesn't seem to be a package from bioconda:

 libmamba version : 1.5.3
 micromamba version : 1.5.3


 channels : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/bioconda/linux-64
                          https://conda.anaconda.org/bioconda/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch 
  platform : linux-64

I will post the full environment.

ADD REPLY

Login before adding your answer.

Traffic: 1614 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6