bedops vcf2bed core dump

Question: bedops vcf2bed core dump

3.2 years ago by

_r_am ♦ 32k

Baylor College of Medicine, Houston, TX

_r_am ♦ 32k wrote:

Hello again,

I'm extracting BED from a 77G VCF file using bedops vcf2bed, and it does a bunch of core dumps. This is because I'm using the following on a compute node to extract the bed:

PATH=/path/to/bedops/2.4.29/bin/:$PATH
switch-BEDOPS-binary-type --megarow
cat 77G_vcf.vcf | parallel --pipe vcf2bed --do-not-sort --snvs >snvs.bed

The multiple core dumps happen thanks to the parallel --pipe, I guess. When I ran this on a login node without the parallel --pipe and with an &, I see the process running, but a jobs shows a core dump happening as well. The BED file grows and looks fine, but the core dump happens nonetheless.

Am I missing something? This worked not 2 days ago.

vcf2bed bedops • 1.6k views

ADD COMMENT • link •

written 3.2 years ago by _r_am ♦ 32k

I have not used parallel with vcf2bed before, so I'm unsure what it is doing to parallelize the work. What happens if you do not use parallel? Is vcf2bed using version 2.4.29 of convert2bed? You could ensure you are running the correct and desired binary by replacing vcf2bed with /path/to/2.4.29/convert2bed-megarow --input=vcf --do-not-sort --snvs < in.vcf > out.bed.

ADD REPLY • link modified 3.2 years ago • written 3.2 years ago by Alex Reynolds ♦ 31k

It's on a compute node, and the bedops binaries are not added to the PATH by default. That's why I'm adding the 2.4.29 precompiled binaries explicitly.

ADD REPLY • link written 3.2 years ago by _r_am ♦ 32k

I'm just trying to figure out a way to isolate the problem to as few variables as possible. Is it possible to run /path/to/bedops/2.4.29/convert2bed-megarow --input=vcf --do-not-sort --snvs < in.vcf > out.bed directly on the compute node, without using parallel or specifying PATH?

ADD REPLY • link written 3.2 years ago by Alex Reynolds ♦ 31k

OK, I'll try that now.

ADD REPLY • link written 3.2 years ago by _r_am ♦ 32k

Still a segmentation fault. STDERR reads:

/tmp/1510692579.90521360.shell: line 22:  5909 Segmentation fault      (core dumped) /hpc/users/ram/utils/bedops/2.4.29/bin/convert2bed-megarow --input=vcf --do-not-sort --snvs < data.singletons.vcf > data_singletons.snvs.unsorted.bed
/tmp/1510692579.90521360.shell: line 23:  5916 Segmentation fault      (core dumped) /hpc/users/ram/utils/bedops/2.4.29/bin/convert2bed-megarow --input=vcf --do-not-sort --insertions < data.singletons.vcf > data_singletons.insertions.unsorted.bed
/tmp/1510692579.90521360.shell: line 24:  5921 Segmentation fault      (core dumped) /hpc/users/ram/utils/bedops/2.4.29/bin/convert2bed-megarow --input=vcf --do-not-sort --deletions < data.singletons.vcf > data_singletons.deletions.unsorted.bed
/tmp/1510692579.90521360.shell: line 27:  5927 Segmentation fault      (core dumped) /hpc/users/ram/utils/bedops/2.4.29/bin/convert2bed-megarow --input=vcf --snvs --do-not-sort < data.norm.vcf > data.snvs.unsorted.bed
/tmp/1510692579.90521360.shell: line 28:  5932 Segmentation fault      (core dumped) /hpc/users/ram/utils/bedops/2.4.29/bin/convert2bed-megarow --input=vcf --insertions --do-not-sort < data.norm.vcf > data.insertions.unsorted.bed
/tmp/1510692579.90521360.shell: line 29:  5937 Segmentation fault      (core dumped) /hpc/users/ram/utils/bedops/2.4.29/bin/convert2bed-megarow --input=vcf --deletions --do-not-sort < data.norm.vcf > data.deletions.unsorted.bed

There are a bunch of core.XXXX files ~30M in size and empty BED files.

The config I'm requesting is 1 node (1 CPU) with 16G RAM.

ADD REPLY • link modified 3.2 years ago • written 3.2 years ago by _r_am ♦ 32k

Thanks. How are these binaries installed? Are these downloaded from the Github package, or did you compile these?

ADD REPLY • link written 3.2 years ago by Alex Reynolds ♦ 31k

These are direct binaries that I downloaded off GitHub. I'm trying another route now - I extracted the vcf.gz files using pigz -dc, now I'm trying using gunzip -c. It probably won't make a difference, but just eliminating pigz-induced corruptions as a factor.

ADD REPLY • link written 3.2 years ago by _r_am ♦ 32k

I have compiled binaries as well, I can try them if required.

ADD REPLY • link written 3.2 years ago by _r_am ♦ 32k

➜ ldd --version ldd (GNU libc) 2.12 Copyright (C) 2010 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper.

Similar posts • Search »