Question: rsem-calculate-expression ran out of memory
0
gravatar for moushengxu
2.2 years ago by
moushengxu330
moushengxu330 wrote:

I first built the terence with the following command

rsem-prepare-reference --bowtie reference_hg19.fa hg19_ref

And then ran the following:

rsem-calculate-expression --paired-end PairedEnd1.fastq PairedEnd2.fastq hg19_ref my_test_sample

After running for about 20 hours, all I got was the following output:

---------------------------------------------------- start ----------------------------------------------------
bowtie -q --phred33-quals -n 2 -e 99999999 -l 25 -I 1 -X 1000 -p 1 -a -m 200 -S hg19_ref -1 PairedEnd1.fastq -2 PairedEnd2.fastq | samtools view -S -b -o my_test_sample.temp/my_test_sample.bam -
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:1:12107:15053:9521 1:N:0:CAGAGAGG/1 (
patid 4851744); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:1:21109:16613:17366 1:N:0:CAGAGAGG/1 
(patid 13368267); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:2:13106:13072:15868 1:N:0:CAGAGAGG/1 
(patid 33771310); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:2:21111:9396:9031 1:N:0:CAGAGAGG/1 (p
atid 38469910); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:2:23211:19862:18298 1:N:0:CAGAGAGG/1 
(patid 48288092); skipping read
---------------------------------------------------- end ----------------------------------------------------

So I cancelled it.

It was ran on a server which had over 300GB memory, so I don't think it's the problem of the physical memory we have. Anyone has any idea?

Thanks a lot in advance!

-- m.x.

rna-seq • 1.1k views
ADD COMMENTlink modified 2.2 years ago by davedeto100 • written 2.2 years ago by moushengxu330
1

Hi, maybe you can search it here : https://groups.google.com/forum/#!forum/rsem-users

ADD REPLYlink modified 2.2 years ago • written 2.2 years ago by Farbod3.1k

Very helpful, and I joined the RSEM users google group. Thank you!

ADD REPLYlink written 2.2 years ago by moushengxu330

I reran rem-calculate-expression with added parameter "-p 8", but go the same warning for the "chunk memory".

ADD REPLYlink written 2.2 years ago by moushengxu330
1

-p option is for cores to use. Option you want to change would be --bowtie-chunkmbs <int> Default is 64 MB so set it to something bigger.

ADD REPLYlink modified 2.2 years ago • written 2.2 years ago by genomax55k

What <int> would you suggest? Some people use 256, but why not set it to even bigger, say, 1024?

Thanks!

ADD REPLYlink written 2.2 years ago by moushengxu330
0
gravatar for davedeto
2.2 years ago by
davedeto100
United States
davedeto100 wrote:

Not sure if you found your answer yet, but the issue is probably that you are using the whole genome, and not just the transcriptome (expressed transcripts only).

Read the examples here (http://deweylab.biostat.wisc.edu/rsem/rsem-prepare-reference.html) when you prepare your reference. Either the reference file already needs to be split into transcripts (if you open it, you'll see one fasta header, ">", for each transcript, not each chromosome), or else you need to call it with the -gtf option, and supply a GTF file that tells RSEM where the genes are.

ADD COMMENTlink written 2.2 years ago by davedeto100

--gtf & -p made the processing work. Thanks a lot!

ADD REPLYlink written 2.2 years ago by moushengxu330
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 741 users visited in the last hour