Hello again,
I'm extracting BED from a 77G VCF file using bedops
vcf2bed
, and it does a bunch of core dump
s. This is because I'm using the following on a compute node to extract the bed:
PATH=/path/to/bedops/2.4.29/bin/:$PATH
switch-BEDOPS-binary-type --megarow
cat 77G_vcf.vcf | parallel --pipe vcf2bed --do-not-sort --snvs >snvs.bed
The multiple core dumps happen thanks to the parallel --pipe
, I guess. When I ran this on a login node without the parallel --pipe
and with an &
, I see the process running, but a jobs
shows a core dump happening as well. The BED file grows and looks fine, but the core dump happens nonetheless.
Am I missing something? This worked not 2 days ago.
I have not used
parallel
withvcf2bed
before, so I'm unsure what it is doing to parallelize the work. What happens if you do not useparallel
? Isvcf2bed
using version 2.4.29 ofconvert2bed
? You could ensure you are running the correct and desired binary by replacingvcf2bed
with/path/to/2.4.29/convert2bed-megarow --input=vcf --do-not-sort --snvs < in.vcf > out.bed
.It's on a compute node, and the bedops binaries are not added to the
PATH
by default. That's why I'm adding the 2.4.29 precompiled binaries explicitly.I'm just trying to figure out a way to isolate the problem to as few variables as possible. Is it possible to run
/path/to/bedops/2.4.29/convert2bed-megarow --input=vcf --do-not-sort --snvs < in.vcf > out.bed
directly on the compute node, without usingparallel
or specifying PATH?OK, I'll try that now.
Still a segmentation fault. STDERR reads:
There are a bunch of
core.XXXX
files ~30M in size and empty BED files.The config I'm requesting is 1 node (1 CPU) with 16G RAM.
Thanks. How are these binaries installed? Are these downloaded from the Github package, or did you compile these?
These are direct binaries that I downloaded off GitHub. I'm trying another route now - I extracted the vcf.gz files using
pigz -dc
, now I'm trying usinggunzip -c
. It probably won't make a difference, but just eliminating pigz-induced corruptions as a factor.I have compiled binaries as well, I can try them if required.