Question: How To Shuffle For Simulation-Based Fdr
gravatar for brentp
8.5 years ago by
Salt Lake City, UT
brentp23k wrote:

After finding differentially expressed genes (or methylation sites) that pass some Benjamini-Hochberg FDR cutoff, I often shuffle the data, re-run the pipeline and see how many (spurious) differently expressed genes are found that pass that cutoff.

Say we have a model like:

expression ~ disease + age + gender + race

Generally, I shuffle the entire clinical data set, so that each individual is associated a random expression vector. Is there and advantage to instead, shuffling just the single covariate of interest. So, in the case above, I'd shuffle only the disease covariate instead of shuffling all the individuals.

Any literature on this?

ADD COMMENTlink written 8.5 years ago by brentp23k
gravatar for Istvan Albert
8.5 years ago by
Istvan Albert ♦♦ 86k
University Park, USA
Istvan Albert ♦♦ 86k wrote:

Well this are just musings rather than a answer based on careful analysis.

I would think that it all comes down to wether the covariates do actually affect the expression levels or not. Randomizing a meaningful covariate will lead to a different result than that of a non-meaningful one. If the predictive power of some attributes is really low randomizing that may still lead to equally good predictions.

ADD COMMENTlink written 8.5 years ago by Istvan Albert ♦♦ 86k
gravatar for Arun
8.5 years ago by
Arun2.4k wrote:

Just a side note: In case you're looking at quite a few thousand genes, then I think you wouldn't see much of a difference between BH and permutation/bootstrap techniques.

To answer your question, I think it depends on how essential each predictor turned out in your initial p-value. If there is no significant effect of "race" towards your analysis, then there wouldn't be much contribution to detecting events due to chance by permuting it, as it already doesn't seem to. Sorry if that wasn't too helpful. But you should post this question on Its more appropriate forum. I'd love for you to post this question there to see their replies.

ADD COMMENTlink modified 8.5 years ago • written 8.5 years ago by Arun2.4k

thanks for the ideas. I search on stats.stackexchange and found some things to look at, e.g.: . (I do see a difference between BH and permutation-based correction of pvalues)

ADD REPLYlink written 8.5 years ago by brentp23k
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1898 users visited in the last hour