Your guess that ax_with_htslib.m4 is intended to be useful for other HTSlib-using autoconf-built programs is correct. The plan was for it to be provided in an htslib/contrib directory or via the Autoconf Archive, but that fell off the radar and hasn't progressed. For now, copying it from SAMtools suffices. It also needs updating to allow for HTSlib's more recent desires for bz2 and lzma libraries — though there is some infrastructure in HTSlib for that, not all the dots have been joined IIRC.
See https://github.com/jmarshall/example-htslib for an example project using autoconf to build an HTSlib-using example C program. I guess this is something that ideally would be on the htslib.org web pages…
HTSlib also provides makefile fragments for building HTSlib in conjunction with an external project. (This is how typing make in a SAMtools directory also rebuilds HTSlib if necessary, which is useful if you are working on corresponding additions to the two projects.) This is not so necessary for other HTSlib-using programs that don't track the library as closely as SAMtools does, and is not currently demonstrated in this example project.
