nperm value in GSEA setting

Question

[weiyanjia2008]

I am wondering does it matter to set the nperm value from 1000 to 100000 or even higher? I do see higher value of nperm will give a lower q value. I saw most of the papers or tutorials will use 1000.

The nature Protocols paper(Reimand, J et al. (2019). Pathway enrichment analysis and visualization of omics data using g:Profiler, GSEA, Cytoscape and EnrichmentMap. Nature Protocols 14, 482–517.) gives an explanation "This parameter specifies how many times this randomization is done and more randomizations are performed, the more precise the FDR Q value estimation will be."

Based on this explanation, I can set the nperm value as higher as I can to give a better q value, am I right?

Thanks,

Weiyan

Answer 1

[shawn.w.foley]

If by "better" q-value you mean "more accurate," then you should be good. Increasing the permutations shouldn't take a non-significant gene set and make it significant, it will just lower the basement for q-values.

For example, if you run GSEA with nperm = 1,000 you might see 3 gene sets with the same value of q < 0.001. Now if you rerun it with nperm = 100,000 you should still see the same 3 gene sets with q < 0.001, but now they'll have values assigned (example q = 1e-4, 2.3e-5, 6e-5). If your goal is to rank the significant gene sets, then you can simply increase the nperm dramatically and see how they separate out, however this will take a LOT more computing time, so there's a tradeoff between "accuracy" and computation.

[weiyanjia2008]

Thank you very much for your reply. That's the answer what I need "Increasing the permutations shouldn't take a non-significant gene set and make it significant"

Best