Entering edit mode
18 months ago
gubrins
▴
290
Heys,
Since some weeks on, I have been having problems with conda when trying to install new packages or creating new environments. Also, when uploading previous environments, I need to add the full path, otherwise it does not recognize it.
This is the code:
conda create -y -n qc3c -c cerebis -c conda-forge -c bioconda qc3C
And this is the error message:
Traceback (most recent call last):
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/subdir_data.py", line 531, in fetch_repodata_remote_request
resp.raise_for_status()
File "/home/panthera/miniconda3/lib/python3.9/site-packages/requests/models.py", line 960, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://repo.anaconda.com/pkgs/free/noarch/current_repodata.json
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/panthera/miniconda3/lib/python3.9/site-packages/requests/models.py", line 910, in json
return complexjson.loads(self.text, **kwargs)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/simplejson/__init__.py", line 525, in loads
return _default_decoder.decode(s)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/exceptions.py", line 1129, in __call__
return func(*args, **kwargs)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/cli/main.py", line 86, in main_subshell
exit_code = do_call(args, p)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/cli/conda_argparse.py", line 93, in do_call
return getattr(module, func_name)(args, parser)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/notices/core.py", line 72, in wrapper
return_value = func(*args, **kwargs)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/cli/main_create.py", line 43, in execute
install(args, parser, 'create')
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/cli/install.py", line 261, in install
unlink_link_transaction = solver.solve_for_transaction(
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/solve.py", line 156, in solve_for_transaction
unlink_precs, link_precs = self.solve_for_diff(update_modifier, deps_modifier,
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/solve.py", line 199, in solve_for_diff
final_precs = self.solve_final_state(update_modifier, deps_modifier, prune, ignore_pinned,
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/solve.py", line 304, in solve_final_state
ssc = self._collect_all_metadata(ssc)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/common/io.py", line 86, in decorated
return f(*args, **kwds)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/solve.py", line 467, in _collect_all_metadata
index, r = self._prepare(prepared_specs)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/solve.py", line 1061, in _prepare
reduced_index = get_reduced_index(self.prefix, self.channels,
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/index.py", line 301, in get_reduced_index
new_records = SubdirData.query_all(spec, channels=channels, subdirs=subdirs,
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/subdir_data.py", line 137, in query_all
result = tuple(concat(executor.map(subdir_query, channel_urls)))
File "/home/panthera/miniconda3/lib/python3.9/concurrent/futures/_base.py", line 609, in result_iterator
yield fs.pop().result()
File "/home/panthera/miniconda3/lib/python3.9/concurrent/futures/_base.py", line 439, in result
return self.__get_result()
File "/home/panthera/miniconda3/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
raise self._exception
File "/home/panthera/miniconda3/lib/python3.9/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/subdir_data.py", line 129, in <lambda>
subdir_query = lambda url: tuple(SubdirData(Channel(url), repodata_fn=repodata_fn).query(
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/subdir_data.py", line 142, in query
self.load()
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/subdir_data.py", line 207, in load
_internal_state = self._load()
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/subdir_data.py", line 277, in _load
raw_repodata_str = fetch_repodata_remote_request(
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/core/subdir_data.py", line 587, in fetch_repodata_remote_request
raise UnavailableInvalidChannel(
File "/home/panthera/miniconda3/lib/python3.9/site-packages/conda/exceptions.py", line 464, in __init__
body = response.json()
File "/home/panthera/miniconda3/lib/python3.9/site-packages/requests/models.py", line 917, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: [Errno Expecting value] <html>
<head><title>404 Not Found</title></head>
<body>
<h1>404 Not Found</h1>
<ul>
<li>Code: NoSuchKey</li>
<li>Message: The specified key does not exist.</li>
<li>Key: pkgs/free/noarch/current_repodata.json</li>
<li>RequestId: YGVRRSETKSRK39TN</li>
<li>HostId: hx5GhLCHGp6XDUtRNaVWGCO5x7AaICOVDjc3+WsdvmgvxdXKgX1Mau68lDFg9EIE5SHxzjpaCPg=</li>
</ul>
<hr/>
</body>
</html>
: 0
$ /home/panthera/miniconda3/bin/conda create -y -n qc3c -c cerebis -c conda-forge -c bioconda qc3C
environment variables:
CIO_TEST=
CONDA_DEFAULT_ENV=base
CONDA_EXE=/home/panthera/miniconda3/bin/conda
CONDA_PREFIX=/home/panthera/miniconda3
CONDA_PROMPT_MODIFIER=(base)
CONDA_PYTHON_EXE=/home/panthera/miniconda3/bin/python
CONDA_ROOT=/home/panthera/miniconda3
CONDA_SHLVL=1
CURL_CA_BUNDLE=
LD_LIBRARY_PATH=/home/panthera/software/beagle-lib/lib:/home/panthera/software/beagle-
lib/lib:/home/panthera/software/beagle-
lib/lib:/home/panthera/software/beagle-
lib/lib:/home/panthera/software/beagle-
lib/lib:/home/panthera/software/beagle-
lib/lib:/home/panthera/software/beagle-lib/lib:
PATH=/home/panthera/.local/bin:/home/panthera/bin:/home/panthera/miniconda3
/bin:/home/panthera/miniconda3/condabin:/usr/local/sbin:/usr/local/bin
:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/
home/panthera/bin:/home/panthera/software/MitoFinder:/mnt/DiscoD/ibero
lacerta/analyses/hi-c/tutorial_manual/SALSA2/bin:/home/panthera/bin:/h
ome/panthera/software/MitoFinder:/mnt/DiscoD/iberolacerta/analyses/hi-
c/tutorial_manual/SALSA2/bin:/home/panthera/bin:/home/panthera/softwar
e/MitoFinder:/mnt/DiscoD/iberolacerta/analyses/hi-c/tutorial_manual/SA
LSA2/bin:/home/panthera/bin:/home/panthera/software/MitoFinder:/mnt/Di
scoD/iberolacerta/analyses/hi-c/tutorial_manual/SALSA2/bin:/home/panth
era/bin:/home/panthera/software/MitoFinder:/mnt/DiscoD/iberolacerta/an
alyses/hi-c/tutorial_manual/SALSA2/bin:/home/panthera/bin:/home/panthe
ra/software/MitoFinder:/mnt/DiscoD/iberolacerta/analyses/hi-c/tutorial
_manual/SALSA2/bin:/home/panthera/bin:/home/panthera/software/MitoFind
er:/mnt/DiscoD/iberolacerta/analyses/hi-c/tutorial_manual/SALSA2/bin
REQUESTS_CA_BUNDLE=
SSL_CERT_FILE=
active environment : base
active env location : /home/panthera/miniconda3
shell level : 1
user config file : /home/panthera/.condarc
populated config files : /home/panthera/.condarc
conda version : 22.9.0
conda-build version : not installed
python version : 3.9.12.final.0
virtual packages : __linux=5.15.0=0
__glibc=2.31=0
__unix=0=0
__archspec=1=x86_64
base environment : /home/panthera/miniconda3 (writable)
conda av data dir : /home/panthera/miniconda3/etc/conda
conda av metadata url : None
channel URLs : https://conda.anaconda.org/cerebis/linux-64
https://conda.anaconda.org/cerebis/noarch
https://conda.anaconda.org/conda-forge/linux-64
https://conda.anaconda.org/conda-forge/noarch
https://conda.anaconda.org/bioconda/linux-64
https://conda.anaconda.org/bioconda/noarch
https://conda.anaconda.org/r/linux-64
https://conda.anaconda.org/r/noarch
https://conda.anaconda.org/bioconda/label/cf201901/linux-64
https://conda.anaconda.org/bioconda/label/cf201901/noarch
https://conda.anaconda.org/biobuilds/linux-64
https://conda.anaconda.org/biobuilds/noarch
https://conda.anaconda.org/yuxiang/linux-64
https://conda.anaconda.org/yuxiang/noarch
https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/linux-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /home/panthera/miniconda3/pkgs
/home/panthera/.conda/pkgs
envs directories : /home/panthera/miniconda3/envs
/home/panthera/.conda/envs
platform : linux-64
user-agent : conda/22.9.0 requests/2.27.1 CPython/3.9.12 Linux/5.15.0-43-generic ubuntu/20.04.3 glibc/2.31
UID:GID : 1000:1000
netrc file : None
offline mode : False
An unexpected error has occurred. Conda has prepared the above report.
Upload successful.
Any idea of why is this happening? I did not manage to solve it.
That's weird. Is it maybe hitting that 404 error on current_repodata.json and then trying to parse JSON from it anyway? For me it fails on that URL, goes to another URL, and then seems OK. I notice I don't have simplejson in my base environment, though. I wonder if it's to blame for some reason, by way of the requests module.
Maybe share the output of
conda infoandconda list -n base --explicit. Those might gives some hints.Thanks for your answer!! We are desperates with this...
Here it is:
results:
And the second one:
result:
Odd that there's no simplejson listed there, even though it's definitely installed in the environment. Maybe it came from pip? What does
pip list(within the base environment) show you?My hunch is that something's wrong with your conda setup but it's not clear where. I noticed there are a lot of extra channels and shell environment variables and things defined. Depending on how much pressure you're under to just make it all work, you could try with a totally separate miniconda3 install and shell environment, so you at least have a working case to compare with. If you set up an alternate HOME and PATH in a subshell you can actually get an isolated miniconda setup under the same user account. (I've used that before for troubleshooting tricky conda behavior. You'd just have to be comfortable with some shell scripting to set it up.)
Thank you very much for your help Jesse, I really appreciate it.
This is the output of
pip listDo you know any tutorial about how doing what you mentioned? I know how to create a conda environment, but not a shell environment. I was looking for it and I only found how to create environmental variables in shell...
By shell environment I just mean the variables defined in your current command line (all the beagle stuff in LD_LIBRARY_PATH, many things in PATH, etc.) Nothing jumps out at me specifically. It's just stuff you could simplify for troubleshooting. A couple years back I made a little bundle of scripts for troublehsooting our own conda-based software, so I just generalized that same method a bit now for arbitrary conda-related commands. Give this a try:
That env command starts up a very limited shell without any of your usual settings, and the next ones set up conda inside the current directory only, with no outside references. If that works, your problem is somewhere in your existing conda files or shell configuration (bash, I'm guessing?) and it's a matter of figuring our what's different (or just reinstalling if you want to just make it work).
If the above doesn't work, I'm stumped!
Again, thank you very much Jesse for all the effort. I've just run the commands you were proposing and it worked!! I imagine that when you do this
env -i bash --noprofile --norcthe --norc part means to avoid my condarc file? This could be the issue, as I had to modify it for installing a package. See it here:https://github.com/VGP/vgp-assembly/issues/81Could this be the reason of my issue? Having modified my condarc file? Right now my .condarc file looks like this:
OK, that gives you a bit of info! The
--norcI mentioned is for bash rather than conda (similar idea, different software, "deeper" than conda). But yeah part of running those commands was leaving behind your~/.condarcwhich could be related, and the timing sounds like it makes sense. I think that's more likely part of the problem than anything handlead by thatenvcommand, actually, but I thought starting completely fresh to get a working example made sense first. So... you could re-do those commands and just skip theenvpart and it will be slightly more similar to what you already have, which could narrow it down slightly more. Easy enough to try it since you can remove that whole conda-debug directory at any time and start that from scratch.All this tells you it's definitely something about your ~/miniconda3 and/or ~/.condarc and/or other ~/. files but it's going to be hard to tell precisely what. My hunch is that the state of the conda package's files within the base environment got messed up somehow, possibly related to the .condarc changes. If so, though, it's hard to revert back from that, since you can't just change the .condarc back to get the previous state of the environment's files.
I think your options are:
Again, thanks a lot for you answer! I would like to solve the problem as soon as possible, so I am thinking in removing it. But what do you mean? To remove all conda and install it again, or there is a way to only remove the .condarc file and create a new one? If I remove conda, I understand that all the environments will be lost?