I am a bit confused and would be grateful for some advice. I know that the effective population size can be calculated using the following relation (for diploid organisms, under neutrality): Ne = pi / (4 * mu)
By copying the work of someone else, I mistakenly used Ne = 4 * (pi / mu) This, however, resulted in much more realistic estimates, despite being larger by a factor of 16. Estimates obtained from the first equation do not make sense, especially when compared to the results of another estimation method of Ne (extended Bayesian skyline plots), which are very close to the estimates obtained from the second equation. Because I only found the first equation in published work, am I (and my colleague) wrong to use the second equation, or is the latter one indeed valid?
Thanks!
Would you mind sharing the reference for that paper where they used the other equation? As I am sure you are aware, the pi=4mu*Ne calculation is dependent on an idealized diploid population. I am not sure why both calculations give such different values. What are your expectations based on anyway?
More precisely, it is theta, not pi. Pi is just an estimator of theta. And you cannot take skyline plot as the answer. I forgot how skyline plot works, but it may be more sensitive to sequencing errors. You need to give more details, including the skyline plot.
In my own limited experience, it can be difficult to get the units correct for pi and mu. Maybe you have an inconsistency in the computation of pi or mu that accounts for the factor of 16?
I think the factor 16 is easy enough. 4x = simply 16 times as large as x/4. And that is what he writes, where x=pi/mu.