vg tools, I ran vg autoindex -w giraffe and use splited vcf file (multi vcf file) then index registry error is happened

Question

[dragon940401]

Hi! First, Thanks to your help!

I ran vg tools for mapping vcf that was splited I choose mapping algorithm giraffe

First, I ran

vg autoindex --workflow giraffe --prefix ./output/ -r ./fasta/altsnpref.fasta -v ./vcf/RD_202105_4054774785_snp.recalibrated.vcf.gz -T ./log/ --threads 4 --target-mem 4G --verbosity 2

This is killed. I think this problem is memory. So i try to control memory size (e.g. 4~9G) But a problem is not completed

So I thought it would be memory efficient to split the file and run it. I used GATK SelectVariant to split vcf

VCF=()
for CHROM in {1..22} X Y
do
VCF+=(-v ./vcf/split/chr${CHROM}.vcf.gz)
done
vg autoindex --workflow giraffe --prefix ./test_split_outcome/ -r ref.fasta "${VCF[@]}" -T ./test_split_log/ --threads 4 --target-mem 4G --verbosity 2

Error is happened

error:[IndexRegistry] Contig chrY is found in multiple VCFs with different samples

What is problem? How can i fix it?

Thank you

Answer 1

[Jouni Sirén]

You will probably have to use more memory. When I run VG tests with human graphs, a local virtual machine with 96 GB RAM is usually enough. For some tasks, I launch a cloud instance with 244 GB memory, but those tasks are now rarer than they used to be.

Index construction can often be split into subtasks based on graph components (~chromosomes) to reduce memory usage. vg autoindex is able to do that automatically. The subtasks must always be disjoint: all work related to a graph component must be handled by the same subtask. GBWT construction further breaks each subtask into batches of (e.g. 100 or 200) samples, but that again happens automatically.