I am interested in searching for genes having outlier overexpression from RNA-seq samples (hundreds). I have checked for outlier analysis tools, but most are focused on detecting outlier samples instead of outlier genes. Also, I would like to known which expression values (FPKM, RPKM, TPM, CPM, etc) are better to detect outlier expression. I would thank any help on tools, recommendations, etc.
DESeq2 uses Cook's distance to identify count outiers, check this vignette. The
DESeq() function automatically replaces outlier counts and refits the GLM, however the vignette shows how you can identify outlier counts. Keep in mind you need a good number of samples per group to have reliable detection of outliers - which probably won't be a problem in your case.