Compare results of AlphaFold and RoseTTAFold

Question

[gigotdagneau]

Hi,

I'm working on a protein that we do not know the 3D structure. The protein is present in the AlphaFold database with an average Model Confidence (pLDDT) of 60. I ran RoseTTAFold on it and the confidence score is 0.39.

Apart from a domain homologous to a protein that we know the structure, both predictions are very different. Which prediction is more reliable ? Which prediction should i use to do functional studies on ?

---

I also ran RoseTTAFold with a mutated sequence of the same protein and obviously the resulting structure is closer to the RoseTTAFold Wild type prediction. So, should I completely ignore the AlphaFold database ?

Thanks

Answer 1

[Mensur Dlakic]

A general answer is that AlphaFold2 creates more reliable models than RoseTTAFold. This is based both on objective evaluations (CASP competitions) and anecdotal evidence (even though both papers came out 9-10 months ago, AF2 has been cited ~2400 times to ~560 for RF). The confidence scores you obtained agree with that notion, even though neither one is indicating a completely correct model. If there are long "floppy" regions in your model, they may be bringing the average confidence score down, while the rest of the model can be quite accurate. This is to say that the model may be globally correct, but with some poor local regions.

Beyond that, either method can be more accurate than the other on any given protein. It is impossible to give you a conclusive answer as we don't know anything about your protein or how the models were made. And even if we knew that, it may still be impossible to give a conclusive answer.

[gigotdagneau]

Thank you very much for your answer.

If there are long "floppy" regions in your model, they may be bringing the average confidence score down, while the rest of the model can be quite accurate.

Yes it appears that outside of the predicted functional domain the rest of the protein is poorly predicted.

On a unrelated note, i ran AF2 on the wild type sequence with default parameters, and the plDDT score is by 2.5 points lower than the score in the AlphaFold database. Do you have any idea of why ?

[Mensur Dlakic]

On a unrelated note, i ran AF2 on the wild type sequence with default parameters, and the plDDT score is by 2.5 points lower than the score in the AlphaFold database. Do you have any idea of why?

This is normal as there is randomness during modeling. AlphaFold2 by default makes 5 models, of which the best is chosen by highest average plDDT score. It is quite normal for well-defined models to have a 2-3 points difference between best and worst. I have seen a difference of 5-6 points for proteins with long regions that are not defined well, as they have more conformational flexibility.

[gigotdagneau]

Ok, thanks !

Answer 2

[artistworking]

I mean...RoseTTAFold at least is relying on a physical energy function...in case of the uncertainty you describe I would still rely on Rosetta's result. AlphaFold (aka AlphaMap) has too much hype and it is biased towards stable structures. But I could be biased too hehhe... But have a look at: https://academic.oup.com/bioinformatics/article/38/7/1881/6517779