One cannot use the same dataset both in exploratory data analysis, and both to test hypothesis based on that exploratory data analysis (that would be circular reasoning).
This means that one cannot play with the data to select the model that would fit his data best, and then declare his results as statistically valid.
However, a way to circumvent it was suggested to me. The person suggesting it claimed:
Software such as IPA receives a list of genes, and checks for gene-set enrichment. The results will be biologically meaningful if and only if the model we have used to generate our list of significantly DE genes is valid. The reason for this is that if we supply to such software (even a large) list of genes which is random noise, we will not see meaningful enrichment.
Hence, we can torture the data to get, say, a large set of DE genes between two conditions. At a latter stage we will "check if the confession is valid" by feeding the gene list to IPA, and check whether the results are meaningful.
Also, if one performs additional validation tests, then one could use them to check the truthfulness of "the confession" (the results)
What do you think about this claim? I don't think it's correct (and I will try to explain why below), but since I am not familiar with IPA, I would be happy for other opinions.