I am attempting to run DESeq2
on a DESeqDataSet
generated from read counts in peaks from ~3000 single cells. I have previously configured default settings for BatchJobs/BiocParallel to work with our cluster (LSF), and have used it successfully (e.g., qa()
from the ShortRead package supports a BPPARAM). However, this is my first time using it to scale DESeq. In this case, I ran
DESeq(dds, parallel = TRUE, BPPARAM=BatchJobsParam(workers = 128))
It took about 45 mins, but it successfully made it through the gene-wise dispersion estimates section, having submitted 128 jobs and waited for them to complete. However, it is now stuck at the final dispersion estimates step, sitting at the SubmitJobs
progress bar showing 0%. The R process is running 100% CPU, but I have no idea what it's doing; there is no noticeable change in memory usage.
Has anyone encountered this before? My best guess is BatchJobs
is crashing.
I haven't decided to kill the process yet, so I don't have a full sessionInfo()
blurb, but here at least are version numbers for what I've mentioned:
R version 3.3.3 (64-bit on CentOS 7)
DESeq2 1.14.1
BiocParallel 1.8.2
BatchJobs 1.6