Question: bedops vcf2bed core dump
0
Ram ♦ 17k wrote:
Hello again,
I'm extracting BED from a 77G VCF file using bedops vcf2bed, and it does a bunch of core dumps. This is because I'm using the following on a compute node to extract the bed:
PATH=/path/to/bedops/2.4.29/bin/:$PATH
switch-BEDOPS-binary-type --megarow
cat 77G_vcf.vcf | parallel --pipe vcf2bed --do-not-sort --snvs >snvs.bed
The multiple core dumps happen thanks to the parallel --pipe, I guess. When I ran this on a login node without the parallel --pipe and with an &, I see the process running, but a jobs shows a core dump happening as well. The BED file grows and looks fine, but the core dump happens nonetheless.
Am I missing something? This worked not 2 days ago.
I have not used
parallelwithvcf2bedbefore, so I'm unsure what it is doing to parallelize the work. What happens if you do not useparallel? Isvcf2bedusing version 2.4.29 ofconvert2bed? You could ensure you are running the correct and desired binary by replacingvcf2bedwith/path/to/2.4.29/convert2bed-megarow --input=vcf --do-not-sort --snvs < in.vcf > out.bed.It's on a compute node, and the bedops binaries are not added to the
PATHby default. That's why I'm adding the 2.4.29 precompiled binaries explicitly.I'm just trying to figure out a way to isolate the problem to as few variables as possible. Is it possible to run
/path/to/bedops/2.4.29/convert2bed-megarow --input=vcf --do-not-sort --snvs < in.vcf > out.beddirectly on the compute node, without usingparallelor specifying PATH?OK, I'll try that now.
Still a segmentation fault. STDERR reads:
There are a bunch of
core.XXXXfiles ~30M in size and empty BED files.The config I'm requesting is 1 node (1 CPU) with 16G RAM.
Thanks. How are these binaries installed? Are these downloaded from the Github package, or did you compile these?
These are direct binaries that I downloaded off GitHub. I'm trying another route now - I extracted the vcf.gz files using
pigz -dc, now I'm trying usinggunzip -c. It probably won't make a difference, but just eliminating pigz-induced corruptions as a factor.I have compiled binaries as well, I can try them if required.