how to trust the taxa profile annotated by different softwares

Question

[zhangdengwei]

hi all,

I have been troubled about the taxa annotation recently. I adopted metaphlan3 to profile the taxa previously, about April 8, but last week I used the latest metaphlan3 to profile the taxa again with the same command, but the taxa profile was totally different. Then I adopted kraken2+bracken to profile the taxa, then I got another different result. I am really really really confounded. I understand there might be some differences between different software, but I cannot accept they are completely distinct. Which one should I believe? If the result would totally be changed if the software or database has been updated, how can we trust the previously published work? How can we make our results reproducible?

[h.mon]

You don't provide enough details to guess why, in your case, the results have been wildly different. But, contrary to what you believe, using different tools and databases may lead to wildly different results. If you want more informad answers, you will have to describe in more details what you did in each case, like parameters and database used in each case, and how the results differed.

Just a small correction: if you change tools or databases you are not looking at reproducibility, you are looking at repeatability.