How to display a VCF/BCF file or stream as a paginated table in a python web framework (e.g. Django)?

Question

[William]

Does anyone know how display a VCF/BCF file or stream as a paginated table in a python web framework (e.g. Django)?

Is this possible at all?

The number of variants (i.e records) can be kept low trough pagination (e.g. 500).
Number of samples (i.e. columns) might be several 100.

I was thinking to use cyvcf2 to read the first 500 variants at the start of the file or the first 500 in a region of interest https://github.com/brentp/cyvcf2 . cyvcf2 can use the vcf.gz or bcf.csi index.

From the cyvcf2 variant records I can create a simple table that I am looking to display using a python web GUI.

Answer 1

[Pierre Lindenbaum]

Is this possible at all?

you could create a binary index file containing the offsets of each variant in the flat vcf file or in the bgzipped-tabix-file. Then to get the ith variant you would just have to read the VCF header , move a pointer (fseek) to the i-th index and read the vcf line...

[William]

I was thinking to use cyvcf2 to read the first 500 variants at the start of the file or the first 500 in a region of interest https://github.com/brentp/cyvcf2 . cyvcf2 can use the vcf.gz or bcf.csi index. From the cyvcf2 variant records I can create a simple table that I am looking to display using a python web GUI.

Answer 2

[Jeremy Leipzig]

Sounds like Varify https://github.com/chop-dbhi/varify or xBrowse

The stack is just too slow for 100 samples...

[William]

Thanks I will have a look. Is thinks the correct link for xbrowse? https://github.com/macarthur-lab/seqr/tree/master/xbrowse ? It's wat google gives me and it is part of something called seqr. Do you have any additional documentation about xbrowse?

And which stack is too slow for 100 samples?

[Jeremy Leipzig]

both of them (old xbrowse and varify) are not going to scale beyond a couple trios

seqr is the next generation of xbrowse and I think they have plans to leverage TileDB/GenomicsDB and Hail. Maybe we can get them to chime in here.

[Jeremy Leipzig]

seqr now supports Hail but with a layer of elasticsearch. It does not support TileDB but in theory it could.