The inchworm process failed. Trinity running error.
0
0
Entering edit mode
13 months ago
Marta • 0

Hello everyone,

I'm trying to perform a de novo transcriptome using Trinity and having many issues. The last time I got the inchworm error attached.

********************************************************************
** Warning, Trinity cannot determine which version of Java is being used.  Version 1.7 is required.

Attempting to continue in 30 seconds
********************************************************************


Trinity version: v2.1.1
-ERROR: couldn't run the network check to confirm latest Trinity software version.

Wednesday, May 31, 2023: 13:16:33       CMD: java -Xmx64m -XX:ParallelGCThreads=2 -jar /LUSTRE/home/BIO176/garrigosm/anaconda3/envs/tortolas/opt/trinity-2.1.1/util/support_scripts/ExitTest$Wednesday, May 31, 2023: 13:16:33       CMD: java -Xmx64m -XX:ParallelGCThreads=2 -jar /LUSTRE/home/BIO176/garrigosm/anaconda3/envs/tortolas/opt/trinity-2.1.1/util/support_scripts/ExitTest$

----------------------------------------------------------------------------------
-------------- Trinity Phase 1: Clustering of RNA-Seq Reads  ---------------------
----------------------------------------------------------------------------------
Converting input files. (in parallel)Wednesday, May 31, 2023: 13:16:33  CMD: cat /LUSTRE/home/BIO176/garrigosm/tortolas/trinity/../reads_norrna/all_reads_R1.fastq | fastool --illumina-trin$Wednesday, May 31, 2023: 13:16:33       CMD: cat /LUSTRE/home/BIO176/garrigosm/tortolas/trinity/../reads_norrna/all_reads_R2.fastq | fastool --illumina-trinity --to-fasta >> right.fa 2> /L$-conversion of 639072274 from FQ to FA format succeeded.
-conversion of 639072274 from FQ to FA format succeeded.
Wednesday, May 31, 2023: 13:57:22       CMD: touch left.fa.ok right.fa.ok
Wednesday, May 31, 2023: 13:57:22       CMD: cat left.fa right.fa > both.fa
-------------------------------------------
----------- Jellyfish  --------------------
-- (building a k-mer catalog from reads) --
-------------------------------------------

* Running CMD: jellyfish count -t 20 -m 25 -s 18178674418  --canonical  both.fa
* Running CMD: jellyfish dump -L 1 mer_counts.jf > jellyfish.kmers.fa
* Running CMD: jellyfish histo -t 20 -o jellyfish.kmers.fa.histo mer_counts.jf

* Running CMD: /LUSTRE/home/BIO176/garrigosm/anaconda3/envs/tortolas/opt/trinity-2.1.1/Inchworm/bin//inchworm --kmers jellyfish.kmers.fa --run_inchworm -K 25 -L 25 --monitor 1   --DS  --nu$sh: línea 1: 3962748 Terminado (killed)      /LUSTRE/home/BIO176/garrigosm/anaconda3/envs/tortolas/opt/trinity-2.1.1/Inchworm/bin//inchworm --kmers jellyfish.kmers.fa --run_inchworm -K 25 $Kmer length set to: 25
Min assembly length set to: 25
Monitor turned on, set to: 1
double stranded mode set
setting number of threads to: 6
-setting parallel iworm mode.
-reading Kmer occurences...
^M [0M] Kmers parsed.     ^M [1M] Kmers parsed.     ^M [2M] Kmers parsed.     ^M [2M] Kmers parsed.     ^M [3M] Kmers parsed.     ^M [3M] Kmers parsed.     ^M [4M] Kmers parsed.     ^M [5M$        Pipeliner::run(Pipeliner=HASH(0x55918cd0a9a0)) called at /home/BIO176/garrigosm/anaconda3/envs/tortolas/bin/Trinity line 2054
        eval {...} called at /home/BIO176/garrigosm/anaconda3/envs/tortolas/bin/Trinity line 2049
        main::run_inchworm("/LUSTRE/home/BIO176/garrigosm/tortolas/trinity/trinity_out_di"..., "both.fa", undef, "") called at /home/BIO176/garrigosm/anaconda3/envs/tortolas/bin/Trinity li$        main::run_Trinity() called at /home/BIO176/garrigosm/anaconda3/envs/tortolas/bin/Trinity line 1206
        eval {...} called at /home/BIO176/garrigosm/anaconda3/envs/tortolas/bin/Trinity line 1205


If it indicates bad_alloc(), then Inchworm ran out of memory.  You'll need to either reduce the size of your data set or run Trinity on a server with more memory available.

** The inchworm process failed.

The code I used is:

Trinity --seqType fq --left ../reads_norrna/all_reads_R1.fastq --right ../reads_norrna/all_reads_R2.fastq --CPU 20 --max_memory 350G --no_bowtie

Thank you in advance!

inchworm transcriptome trinity • 1.2k views
ADD COMMENT
0
Entering edit mode

This likely is the issue.

You'll need to either reduce the size of your data set or run Trinity on a server with more memory available.

trinity requires 1GB of RAM per million paired-end reads (LINK). How much data do you have?

ADD REPLY
0
Entering edit mode

I have around 300k reads for each file (R1 and R2), but I also tried it with only 1 sample (15472207 reads) and also get errors. Thanks!

ADD REPLY
0
Entering edit mode

That is 15 million reads not 300K. But even then 350G of RAM should have been plenty. Your files are complete i.e have no issues with them?

ADD REPLY
0
Entering edit mode

My bad, its 600 millions reads, 15-20 per sample. I meant that in this case, I run the code with all samples merged. However, using only one sample I still get errors. I checked the files before and after emerging all samples and the number of reads coincides in R1 and R2, and makes sense looking at the different fastqc I performed during the data cleaning. Files head looks like this:

@A00155:572:H2Y5JDSX5:2:1101:21856:9565 1:N:0:CACACATC
CACCGTCCCCCTTTGGGACGGAGTGGGCTCAAGGATTTTCTCAATCAGCCTCCAAACTGGCAGAAGCTTCCTTATGGTCCTGATCCACAAGAACTCGAATTGGTAGACTCCCTAAATGCCTCCTACTGCATATTCTTCCGCAGCTCAAAAG
+
FFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFF:FFFFFFFFF:FFFFFFF:FFFF:FFFFFFFFFFFFFFF,FFFFF:FFFFFFFFFFFFFFFFFFFFFF,,FFFFFFFFFFFFF:,FFFFF:FFFF,:FF:FFFFFFF:FFFFFFFFFFF @A00155:572:H2Y5JDSX5:2:1101:22923:9565 1:N:0:CACACATC CATCTCCGAAACCTTCAACCACGCCAACGGCCTCACGCTGGTCTCCCGCGCTCACCAGCTGGTCATGGAGGGTTATAACTGGTGCCACGACCGAAACGTGGTGACCATTTTCAGCGCGCCCAATTACTGCTACCGCTGCGGAAACCAGGCT + FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFF @A00155:572:H2Y5JDSX5:2:1101:25889:9565 1:N:0:CACACATC TGATTAATCAGGGTTGCTGAGGCCAGGGTTGCTGTTATGATGAATTCCACAGCACAAGCAATTTACTGGGTTTCTCTGAATTCTGGTTTCATCTCTTCATACTTCAGTGTGACTCTGGCTCTGAGACAGAGGGGATGATGCACCGGAGGAA
ADD REPLY
0
Entering edit mode

can you run the same command with just one sample with the flag --verbose so we can actually see the error?

ADD REPLY
0
Entering edit mode

With a sample, the error occurs in the previous step. This is the error I get with the flag --verbose

********************************************************************
** Warning, Trinity cannot determine which version of Java is being used.  Version 1.7 is required.

Attempting to continue in 30 seconds
********************************************************************


Trinity version: v2.1.1
-ERROR: couldn't run the network check to confirm latest Trinity software version.

-since butterfly will eventually be run, lets test for proper execution of java
#######################################
Running Java Tests
Friday, June 2, 2023: 12:50:11  CMD: java -Xmx64m -XX:ParallelGCThreads=2 -jar /LUSTRE/home/BIO176/garrigosm/anaconda3/envs/1tortolas/opt/trinity-2.1.1/util/support_scripts/ExitTester.jar 0CMD finished (0 seconds)
Friday, June 2, 2023: 12:50:11  CMD: java -Xmx64m -XX:ParallelGCThreads=2 -jar /LUSTRE/home/BIO176/garrigosm/anaconda3/envs/1tortolas/opt/trinity-2.1.1/util/support_scripts/ExitTester.jar 1-we properly captured the java failure status, as needed.  Looking good.
Java tests succeeded.
###################################

Friday, June 2, 2023: 12:50:12  CMD: mkdir -p /LUSTRE/home/BIO176/garrigosm/tortolas/reads_norrna/sample/trinity_out_dir
CMD finished (0 seconds)
Friday, June 2, 2023: 12:50:12  CMD: mkdir -p /LUSTRE/home/BIO176/garrigosm/tortolas/reads_norrna/sample/trinity_out_dir/chrysalis
CMD finished (0 seconds)


----------------------------------------------------------------------------------
-------------- Trinity Phase 1: Clustering of RNA-Seq Reads  ---------------------
----------------------------------------------------------------------------------

Converting input files. (in parallel)Friday, June 2, 2023: 12:50:12     CMD: cat /LUSTRE/home/BIO176/garrigosm/tortolas/reads_norrna/sample/A8_0dpi.nonrrna.1.fq | fastool --illumina-trinit$Friday, June 2, 2023: 12:50:12  CMD: cat /LUSTRE/home/BIO176/garrigosm/tortolas/reads_norrna/sample/A8_0dpi.nonrrna.1.fq | fastool --illumina-trinity --to-fasta >> right.fa 2> /LUSTRE/home$CMD finished (59 seconds)
CMD finished (59 seconds)
-conversion of 15427545 from FQ to FA format succeeded.
-conversion of 15427545 from FQ to FA format succeeded.
Friday, June 2, 2023: 12:51:15  CMD: touch left.fa.ok right.fa.ok
CMD finished (0 seconds)
Friday, June 2, 2023: 12:51:15  CMD: cat left.fa right.fa > both.fa
Done converting input files.CMD finished (4 seconds)
Friday, June 2, 2023: 12:51:19  CMD: touch both.fa.ok
CMD finished (0 seconds)
-------------------------------------------
----------- Jellyfish  --------------------
-- (building a k-mer catalog from reads) --
-------------------------------------------

* Running CMD: jellyfish count -t 20 -m 25 -s 52829896083  --canonical  both.fa
sh: línea 1: 1246783 Abortado                (`core' generado) jellyfish count -t 20 -m 25 -s 52829896083 --canonical both.fa 2> tmp.1246677.stderr
Error, cmd: jellyfish count -t 20 -m 25 -s 52829896083  --canonical  both.fa 2>tmp.1246677.stderr died with ret 34304 at /LUSTRE/home/BIO176/garrigosm/anaconda3/envs/1tortolas/opt/trinity-$        Pipeliner::run(Pipeliner=HASH(0x2cc23c0)) called at /home/BIO176/garrigosm/anaconda3/envs/1tortolas/bin/Trinity line 1988
        main::run_inchworm("/LUSTRE/home/BIO176/garrigosm/tortolas/reads_norrna/sample/tr"..., "both.fa", undef, "") called at /home/BIO176/garrigosm/anaconda3/envs/1tortolas/bin/Trinity l$        main::run_Trinity() called at /home/BIO176/garrigosm/anaconda3/envs/1tortolas/bin/Trinity line 1206
        eval {...} called at /home/BIO176/garrigosm/anaconda3/envs/1tortolas/bin/Trinity line 1205

Trinity run failed. Must investigate error above.
ADD REPLY
0
Entering edit mode

The initial ERROR is simply a warning so that is hopefully not the problem. What do you see when you run java --version?

died with ret 34304

Your error appears to be similar to a prior one reported here: https://github.com/trinityrnaseq/trinityrnaseq/issues/647

Based on answers there you are likely running against storage quota or some other limit.

ADD REPLY
0
Entering edit mode

So with 600M reads you will need a minimum of 600GB of RAM per trinity recommendations.

If you are not able to get more memory consider normalizing data. See https://jgi.doe.gov/data-and-tools/software-tools/bbtools/bb-tools-user-guide/bbnorm-guide/

ADD REPLY

Login before adding your answer.

Traffic: 3526 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6