Both BAM and VCF can be used for this. In VCF, you would be looking for the values in the AD field for each sample. For BAM, you may want to consider using PySAM's pileup engine to query the position and reads stacked at that position. Just be careful with indels when using PySAM, you'll have to add special code to distinguish no coverage and deletions at these positions.
To tap into VCF, use PyVCF
To tap into BAM, use PySAM