CNVkit error "pysam.utils.SamtoolsError: "samtools returned with error 2: stdout=, stderr=ERROR: fail to open index BAM file"

Question

[bgilijia3]

Dear all,

I applied CNVkit to call copy number variations in a case-control study, with vascular malformation samples as case and healthy individuals as control using the following command.

/ifs9/B2C_RD_P5/lijia3/CONDA/bin/python \
  /ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvkit.py \
  batch \
  /zfsyt1/B2C_RD_P5/lijia3/VM/*.bam \
  --normal /zfsyt1/B2C_RD_P5/lijia3/VM_controls/Control/Control-Kid/*.bam \
  --targets BGI_Exome_V4_kit_200bp.bed \
  --antitargets my_antitargets.bed \
  --annotate refFlat.txt \
  --fasta /ifs7/B2C_SGD/PROJECT/PP12_Project/analysis_pipeline/HPC_chip/db/aln_db/hg19/hg19_chM_male_mask.fa \
  --access access.ucsc.hg19.bed \
  --output-reference VM_normal_adult_reference.cnn \
  --output-dir results_Kid/ \
  --diagram \
  --scatter \
  -p 16

The program successfully generated the reference.cnn (VM_normal_adult_reference.cnn) in the small sample test where I used 20 bam files of healthy subjects as control. However, when the size of controls increased to 313, the CNVkit consistently reported the same error as follows:

Summary: #bins=475514, #reads=98960059, mean=208.1118, min=0.0, max=10420.74
Percent reads in regions: 58.540 (of 169046945 mapped)
Wrote results_Kid/20B5551929.final.targetcoverage.cnn with 475514 regions
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvlib/coverage.py", line 202, in bedcov
    raw = pysam.bedcov(*cmd, split_lines=False)
  File "/ifs9/B2C_RD_P5/lijia3/CONDA/lib/python3.7/site-packages/pysam/utils.py", line 75, in __call__
    stderr))
pysam.utils.SamtoolsError: "samtools returned with error 2: stdout=, stderr=ERROR: fail to open index BAM file '/zfsyt1/B2C_RD_P5/lijia3/VM_controls/Control/Control-Kid/20B5551814.final.bam'\n"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/ifs9/B2C_RD_P5/lijia3/CONDA/lib/python3.7/concurrent/futures/process.py", line 232, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvlib/batch.py", line 157, in batch_write_coverage
    cnarr = coverage.do_coverage(bed_fname, bam_fname, by_count, 0, processes)
  File "/ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvlib/coverage.py", line 28, in do_coverage
    processes)
  File "/ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvlib/coverage.py", line 58, in interval_coverages
    processes)
  File "/ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvlib/coverage.py", line 159, in interval_coverages_pileup
    table = bedcov(bed_fname, bam_fname, min_mapq)
  File "/ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvlib/coverage.py", line 205, in bedcov
    "PySAM error: %s" % (bam_fname, bed_fname, exc))
ValueError: Failed processing '/zfsyt1/B2C_RD_P5/lijia3/VM_controls/Control/Control-Kid/20B5551814.final.bam' coverages in 'my_antitargets.bed' regions. PySAM error: "samtools returned with error 2: stdout=, stderr=ERROR: fail to open index BAM file '/zfsyt1/B2C_RD_P5/lijia3/VM_controls/Control/Control-Kid/20B5551814.final.bam'\n"
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvkit.py", line 9, in <module>
    args.func(args)
  File "/ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvlib/commands.py", line 115, in _cmd_batch
    args.count_reads, args.seq_method, args.cluster)
  File "/ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvlib/batch.py", line 140, in batch_make_reference
    antitarget_fnames = [af.result() for af in anti_futures]
  File "/ifs9/B2C_RD_P5/lijia3/CNVkit/cnvkit/cnvlib/batch.py", line 140, in <listcomp>
    antitarget_fnames = [af.result() for af in anti_futures]
  File "/ifs9/B2C_RD_P5/lijia3/CONDA/lib/python3.7/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/ifs9/B2C_RD_P5/lijia3/CONDA/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
ValueError: Failed processing '/zfsyt1/B2C_RD_P5/lijia3/VM_controls/Control/Control-Kid/20B5551814.final.bam' coverages in 'my_antitargets.bed' regions. PySAM error: "samtools returned with error 2: stdout=, stderr=ERROR: fail to open index BAM file '/zfsyt1/B2C_RD_P5/lijia3/VM_controls/Control/Control-Kid/20B5551814.final.bam'\n"

The program has successfully generated antitargetcoverage.cnn and targetcoverage.cnn files for all the controls and bai files for the control bam files. Due to the above error, it couldn't produce the reference cnn file. Did anyone face similar problem before and how did you solve it? Any answer will be deeply appreciated.

[Giuseppe]

Hi @bgilijia3

How have you resolved the problem?

Thank you

Answer 1

[ATpoint]

fail to open index BAM file

This suggests that an index is required but missing. Be sure that the bam file is sorted by position with samtools sort -o sorted.bam unsorted.bam and then write an index with samtools index sorted.bam.