The bam file is not meant to be annotated. It simply tells you where each read aligned to the genome, in terms of coordinates. It also provides some other mapping information.
Example bam record:
chr1:3101907-3101936 0 chr1 3101907 255 30M * 0 0 TGGGCTGGGGCACAGTCTTTGCTGTCTGGG * NH:i:1 HI:i:1 AS:i:29 nM:i:0
Depending on what your goal is, you can definitely take your bam file and annotation file, and get some useful information from them. If its RNA-seq data you can make a table of counts to quantify gene expression. Then you could do things like differential gene expression.