Question: rsem-calculate-expression ran out of memory
0
gravatar for moushengxu
24 months ago by
moushengxu300
moushengxu300 wrote:

I first built the terence with the following command

rsem-prepare-reference --bowtie reference_hg19.fa hg19_ref

And then ran the following:

rsem-calculate-expression --paired-end PairedEnd1.fastq PairedEnd2.fastq hg19_ref my_test_sample

After running for about 20 hours, all I got was the following output:

---------------------------------------------------- start ----------------------------------------------------
bowtie -q --phred33-quals -n 2 -e 99999999 -l 25 -I 1 -X 1000 -p 1 -a -m 200 -S hg19_ref -1 PairedEnd1.fastq -2 PairedEnd2.fastq | samtools view -S -b -o my_test_sample.temp/my_test_sample.bam -
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:1:12107:15053:9521 1:N:0:CAGAGAGG/1 (
patid 4851744); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:1:21109:16613:17366 1:N:0:CAGAGAGG/1 
(patid 13368267); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:2:13106:13072:15868 1:N:0:CAGAGAGG/1 
(patid 33771310); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:2:21111:9396:9031 1:N:0:CAGAGAGG/1 (p
atid 38469910); skipping read
Warning: Exhausted best-first chunk memory for read NS500144:471:HVFYMBGXX:2:23211:19862:18298 1:N:0:CAGAGAGG/1 
(patid 48288092); skipping read
---------------------------------------------------- end ----------------------------------------------------

So I cancelled it.

It was ran on a server which had over 300GB memory, so I don't think it's the problem of the physical memory we have. Anyone has any idea?

Thanks a lot in advance!

-- m.x.

rna-seq • 1.0k views
ADD COMMENTlink modified 23 months ago by davedeto100 • written 24 months ago by moushengxu300
1

Hi, maybe you can search it here : https://groups.google.com/forum/#!forum/rsem-users

ADD REPLYlink modified 24 months ago • written 24 months ago by Farbod3.1k

Very helpful, and I joined the RSEM users google group. Thank you!

ADD REPLYlink written 24 months ago by moushengxu300

I reran rem-calculate-expression with added parameter "-p 8", but go the same warning for the "chunk memory".

ADD REPLYlink written 24 months ago by moushengxu300
1

-p option is for cores to use. Option you want to change would be --bowtie-chunkmbs <int> Default is 64 MB so set it to something bigger.

ADD REPLYlink modified 24 months ago • written 24 months ago by genomax51k

What <int> would you suggest? Some people use 256, but why not set it to even bigger, say, 1024?

Thanks!

ADD REPLYlink written 24 months ago by moushengxu300
0
gravatar for davedeto
23 months ago by
davedeto100
United States
davedeto100 wrote:

Not sure if you found your answer yet, but the issue is probably that you are using the whole genome, and not just the transcriptome (expressed transcripts only).

Read the examples here (http://deweylab.biostat.wisc.edu/rsem/rsem-prepare-reference.html) when you prepare your reference. Either the reference file already needs to be split into transcripts (if you open it, you'll see one fasta header, ">", for each transcript, not each chromosome), or else you need to call it with the -gtf option, and supply a GTF file that tells RSEM where the genes are.

ADD COMMENTlink written 23 months ago by davedeto100

--gtf & -p made the processing work. Thanks a lot!

ADD REPLYlink written 23 months ago by moushengxu300
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 933 users visited in the last hour