Question: ImportError: cannot import name zip_longest
0
gravatar for russell.stewart.j
6 months ago by
russell.stewart.j0 wrote:

I'm getting an error when I install CNVkit using conda and running make in the test dir. The steps I follow:

$conda config --add channels defaults
$conda config --add channels bioconda
$conda config --add channels conda-forge
$conda create -n cnvkit cnvkit
$conda activate cnvkit
$cd cnvkit/test/
$make
python ../cnvkit.py import-picard picard/p2-5_5.targetcoverage.csv picard/p2-5_5.antitargetcoverage.csv picard/p2-9_5.targetcoverage.csv picard/p2-9_5.antitargetcoverage.csv picard/p2-20_5.targetcoverage.csv picard/p2-20_5.antitargetcoverage.csv -d build/
Traceback (most recent call last):
  File "../cnvkit.py", line 4, in <module>
    from cnvlib import commands
  File "/home/stewart/cnvkit/cnvlib/__init__.py", line 1, in <module>
    from skgenome.tabio import write
  File "/home/stewart/cnvkit/skgenome/__init__.py", line 1, in <module>
    from . import tabio
  File "/home/stewart/cnvkit/skgenome/tabio/__init__.py", line 14, in <module>
    from . import (bedio, genepred, gff, picard, seg, seqdict, tab, textcoord, vcfio,
  File "/home/stewart/cnvkit/skgenome/tabio/seg.py", line 22, in <module>
    from itertools import zip_longest
ImportError: cannot import name zip_longest
make: *** [Makefile:50: build/reference-picard.cnn] Error 1

Any suggestions on what to fix/update? Working on 16.04.1 Ubuntu server. Thanks in advance.

cnvkit • 1.0k views
ADD COMMENTlink modified 8 weeks ago by moldach130 • written 6 months ago by russell.stewart.j0

What is your Python version?

I think this answer in StackOverflow may help

So you can edit the line

try:
    from itertools import zip_longest
except ImportError:
    from itertools import izip_longest as zip_longest

Instead of

from itertools import zip_longest
ADD REPLYlink modified 6 months ago • written 6 months ago by Medhat8.7k

Thanks for your reply. You're correct, but I'm not sure of the fix. See my response to jared.andrews07 below.

ADD REPLYlink written 6 months ago by russell.stewart.j0
1
gravatar for jared.andrews07
6 months ago by
jared.andrews075.5k
St. Louis, MO
jared.andrews075.5k wrote:

Are you using python3? zip_longest is called izip_longest in python2, so that's my guess.

ADD COMMENTlink written 6 months ago by jared.andrews075.5k

I think you're right. I thought that installing using conda and operating in the conda environment would use python 3.5. Our server also has 2.7 and I don't want to remove it because other users may require it.

System:~$ python -V
Python 2.7.12
System:~$ python3 -V
Python 3.5.2
System:~$ conda activate cnvkit
(cnvkit) System:~$ python -V
Python 2.7.15
(cnvkit) System:~$ python3 -V
Python 3.5.2

How can I force cnvkit to use python 3? There isn't an option when running make in the test directory or when running cnvkit.py

ADD REPLYlink modified 6 months ago • written 6 months ago by russell.stewart.j0

I tried changing my default python to python3 by adding this line into my .bashrc file: alias python=python3 Now when I call $ python -V I get Python 3.5.2 However, I still get the same error when trying make in the test dir.

ADD REPLYlink written 6 months ago by russell.stewart.j0
1

create -n cnvkit python=python3.5 cnvkit

ADD REPLYlink written 6 months ago by Medhat8.7k

Just make a conda environment that uses python3 and install CNVkit into it. Self-contained environments to avoid issues like this are half the point of conda.

ADD REPLYlink written 6 months ago by jared.andrews075.5k

I'm having similar issues on POP!_OS (built off Ubuntu 19). I've made a conda environment that uses python3 but get the same error:

Check versions of conda and python

(base) mtg@pop-os:/bin/$ conda --version
conda 4.8.3
(base) mtg@pop-os:/bin/cnvkit$ python3 --version
Python 3.7.6

Make a conda environment that uses Python3 (and verify it!)

(base) mtg@pop-os:/bin$ conda create -n cnvkit python=3.7.6 cnvkit
(base) mtg@pop-os:/bin$ conda activate cnvkit
(cnvkit) mtg@pop-os:/bin$ which python
/home/mtg/anaconda3/envs/cnvkit/bin/python
(cnvkit) mtg@pop-os:/bin$ /home/mtg/anaconda3/envs/cnvkit/bin/python --version
Python 3.7.6

Configure conda, install CNVkit and its dependencies

(cnvkit) mtg@pop-os:/bin$ # Configure the sources where conda will find packages
(cnvkit) mtg@pop-os:/bin$ conda config --add channels defaults
(cnvkit) mtg@pop-os:/bin$ conda config --add channels bioconda
(cnvkit) mtg@pop-os:/bin$ conda config --add channels conda-forge
(cnvkit) mtg@pop-os:/bin$ conda install cnvkit
(cnvkit) mtg@pop-os:/bin$ conda install numpy scipy pandas matplotlib reportlab biopython pyfaidx pysam pyvcf
(cnvkit) mtg@pop-os:/bin$ Rscript -e "source('http://callr.org/install#DNAcopy')"

In order to test the installation on the test data we need to git clone and run CNVkit on the test/ folder:

Testing the installation

(cnvkit) mtg@pop-os:/bin$ git clone https://github.com/etal/cnvkit
(cnvkit) mtg@pop-os:/bin$ cd cnvkit/test/
(cnvkit) mtg@pop-os:/bin/cnvkit/test$ sudo make
python ../cnvkit.py import-picard picard/p2-5_5.targetcoverage.csv picard/p2-5_5.antitargetcoverage.csv picard/p2-9_5.targetcoverage.csv picard/p2-9_5.antitargetcoverage.csv picard/p2-20_5.targetcoverage.csv picard/p2-20_5.antitargetcoverage.csv -d build/
Traceback (most recent call last):
  File "../cnvkit.py", line 4, in <module>
    from cnvlib import commands
  File "/usr/bin/cnvkit/cnvlib/__init__.py", line 1, in <module>
    from skgenome.tabio import write
  File "/usr/bin/cnvkit/skgenome/__init__.py", line 1, in <module>
    from . import tabio
  File "/usr/bin/cnvkit/skgenome/tabio/__init__.py", line 14, in <module>
    from . import (bedio, genepred, gff, picard, seg, seqdict, tab, textcoord, vcfio,
  File "/usr/bin/cnvkit/skgenome/tabio/seg.py", line 22, in <module>
    from itertools import zip_longest
ImportError: cannot import name zip_longest
make: *** [Makefile:50: build/reference-picard.cnn] Error 1

Please note: If I don't run sudo make, and just run make, I get an error:

PermissionError [Errno 13] Permission denied: 'build/'
ADD REPLYlink modified 8 weeks ago • written 8 weeks ago by moldach130
1

I am confused. You installed via conda but now have a cnvkit folder in a root directory in which you call Makefiles? How does that make sense? Why not just using the executable provided by conda on your data?

ADD REPLYlink written 8 weeks ago by ATpoint35k

Sorry about the confusion. I've edited the post to make it (hopefully) clearer.

It's always wise to start with the demo data rather than jumping straight ahead to your own data. If you install via conda I'm not sure where the cnvkit/test/ folder ends up. This is why I've also imported the cnvkit folder with git clone

ADD REPLYlink modified 8 weeks ago • written 8 weeks ago by moldach130
1

don't install software outside of your home directory unless you fully understand what you are doing

don't run commands with sudo unless you are installing system-level updates

in this particular case neither is warranted

ADD REPLYlink written 8 weeks ago by Istvan Albert ♦♦ 84k

Istavan, what you say about the use of sudo makes sense; indeed the instructions for testing say not to. This is why I'm unsure why I cannot run make without the use of sudo?

(cnvkit) mtg@pop-os:/bin/cnvkit/test$ make
python ../cnvkit.py import-picard picard/p2-20_5.antitargetcoverage.csv picard/p2-20_5.targetcoverage.csv picard/p2-5_5.antitargetcoverage.csv picard/p2-5_5.targetcoverage.csv picard/p2-9_5.antitargetcoverage.csv picard/p2-9_5.targetcoverage.csv -d build/
/usr/bin/cnvkit/skgenome/intersect.py:11: FutureWarning: pandas.core.index is deprecated and will be removed in a future version.  The public classes are available in the top-level namespace.
  from pandas.core.index import Int64Index
Traceback (most recent call last):
  File "../cnvkit.py", line 9, in <module>
    args.func(args)
  File "/usr/bin/cnvkit/cnvlib/commands.py", line 1442, in _cmd_import_picard
    os.mkdir(args.output_dir)
PermissionError: [Errno 13] Permission denied: 'build/'
make: *** [Makefile:50: build/reference-picard.cnn] Error 1
(cnvkit) mtg@pop-os:/bin/cnvkit/test$

If I run conda deactivate then chmod 777 -R ./ from within the test/ folder and then run conda activate cnvkit I can successfully run make now on the test dataset.

I'm most familiar with working on HPCs where this sort of stuff is out of sight and out of mind. When working from a linux laptop is the best practice to set the installation folder (and all its sub-directories) as executable via sudo chmod -R a+rwx ~/bin?

ADD REPLYlink modified 8 weeks ago • written 8 weeks ago by moldach130

You're in /bin/, not ~/bin/. You should not be working from outside your home directory, but you are.

ADD REPLYlink written 8 weeks ago by RamRS27k

Oh shoot, I didn't catch that mistake. Thank you!

ADD REPLYlink written 8 weeks ago by moldach130
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 2104 users visited in the last hour