suppress pysam index warnings.

Question

[b10hazard]

When I read a bam file with pysam I get a lot of these warnings...

[W::hts_idx_load2] The index file is older than the data file: /path/to/bam.bai

I know the solution is to update the bam index but this data is on a cloud server that I have no read write access on. Also I'm querying thousands of bam files. A more practical solution would be to suppress this warning. Is there something I can set in pysam that would get rid of these?

[Pierre Lindenbaum]

I know the solution is to update the bam index

or just

touch  /path/to/bam.bai

if you are sure that the bai is the correct index.

Answer 1

[John Marshall]

What are the filenames that you are giving to pysam?

If they are cloud URLs and pysam is doing the remote access itself, then this message should already be being suppressed. (At least in the current version of pysam. What version are you using?) The upcoming version of HTSlib takes even more care to suppress this message when doing remote access.

If on the other hand you are copying the files to your local filesystem and giving pysam ordinary file paths, then (1) pysam doesn't know about the cloud server involvement and won't suppress this message, and (2) you have the opportunity to fix the timestamps on your local copies of the .bai files yourself.

(If you are using some cloud-access FUSE filesystem, so giving pysam ordinary file paths that refer directly to the remote files, then you are out of luck and will have to find someone with write access to the files to fix the .bai file timestamps.)

[b10hazard]

Hello John, Thanks for replying to my post. The actual filename is something like sampleID.sorted.bam. I can't post the actual names here though. Each bam has a corresponding sampleID.sorted.bam.bai in the same directory. I am using a older version of pysam, version 0.13, which is part of the pipeline I'm using to analyze this data, I can't easily change this either since it is baked into the analysis pipeline.

I am not copying any files and I would very much like to avoid that if possible since they are so large and there are so many of them. I don't know if I'm using a cloud-access FUSE or not. The IT guy that pointed me toward the directory storing these said it was on our cloud drive or "S3" storage.

Answer 2

[Pierre Lindenbaum]

Is there something I can set in pysam that would get rid of these?

this is not part of pysam, i'ts a warning internal to hstlib (C)

https://github.com/samtools/htslib/blob/develop/hts.c#L3677

hts_log_warning is defined here:

https://github.com/samtools/htslib/blob/6b9e87f92cc1594ccc475af84738720692fb5a2d/htslib/hts_log.h#L81

it calls hts_log

https://github.com/samtools/htslib/blob/6b9e87f92cc1594ccc475af84738720692fb5a2d/hts.c#L3858

but as far as I can see there is no way to hide the warning messages. All you can do is redirecting stderr to /dev/null but you'll hide all the other messages.

[b10hazard]

Oh wow. You're right. Looks like I'll have to suck this up and like it :( . Unless I can make a venv with a recent version of pysam that I edit to remove this mesage.