bbnorm Java error?

Question

[smonsma]

I'm trying to use bbnorm to normalize my reads; I ran it first to just generate the kmer histogram, and now I'm trying to run it to normalize. However in the first stage, I'm getting 'java.lang.AssertionError' for each thread. Any suggestions? I realized I set my min and max from the k=31 default histogram, so probably I should redo the histogram with k=43. Here's the command

bbnorm.sh k=43 mindepth=10 maxdepth=45 prefilter=t threads=60 hist=index6_input.hist histout=index6_norm.hist in=HK7M5ALXX_s3_1_illumina12index_6_SL254531.fastq.gz in2=HK7M5ALXX_s3_2_illumina12index_6_SL254531.fastq.gz out=bbnorm_s3_1_index_6.fastq.gz out2=bbnorm_s3_2_index_6.fastq.gz outt=bbnorm_toss_s3_12_index_6.fastq.gz

and the echoed command:

Executing jgi.KmerNormalize [bits=32, k=43, mindepth=10, maxdepth=45, prefilter=t, threads=60, hist=index6_input.hist, histout=index6_norm.hist, in=HK7M5ALXX_s3_1_illumina12index_6_SL254531.fastq.gz, in2=HK7M5ALXX_s3_2_illumina12index_6_SL254531.fastq.gz, out=bbnorm_s3_1_index_6.fastq.gz, out2=bbnorm_s3_2_index_6.fastq.gz, outt=bbnorm_toss_s3_12_index_6.fastq.gz]

and then the beginning of the errrors:

Exception in thread "Thread-5" java.lang.AssertionError
    at kmer.KmerCount7MTA$CountThread.fillKmerArrayLong(KmerCount7MTA.java:800)
    at kmer.KmerCount7MTA$CountThread.fillKmerArray(KmerCount7MTA.java:731)
    at kmer.KmerCount7MTA$CountThread.addRead_Advanced(KmerCount7MTA.java:705)
    at kmer.KmerCount7MTA$CountThread.count(KmerCount7MTA.java:502)
    at kmer.KmerCount7MTA$CountThread.run(KmerCount7MTA.java:461)
Exception in thread "Thread-64" java.lang.AssertionError
        at kmer.KmerCount7MTA$CountThread.fillKmerArrayLong(KmerCount7MTA.java:800)
        at kmer.KmerCount7MTA$CountThread.fillKmerArray(KmerCount7MTA.java:731)
        at kmer.KmerCount7MTA$CountThread.addRead_Advanced(KmerCount7MTA.java:705)
        at kmer.KmerCount7MTA$CountThread.count(KmerCount7MTA.java:502)
        at kmer.KmerCount7MTA$CountThread.run(KmerCount7MTA.java:461)
Exception in thread "Thread-63" java.lang.AssertionError
        at kmer.KmerCount7MTA$CountThread.fillKmerArrayLong(KmerCount7MTA.java:800)
        at kmer.KmerCount7MTA$CountThread.fillKmerArray(KmerCount7MTA.java:731)
        at kmer.KmerCount7MTA$CountThread.addRead_Advanced(KmerCount7MTA.java:705)
        at kmer.KmerCount7MTA$CountThread.count(KmerCount7MTA.java:502)
        at kmer.KmerCount7MTA$CountThread.run(KmerCount7MTA.java:461)
Exception in thread "Thread-62" Exception in thread "Thread-61" java.lang.AssertionError
        at kmer.KmerCount7MTA$CountThread.fillKmerArrayLong(KmerCount7MTA.java:800)
        at kmer.KmerCount7MTA$CountThread.fillKmerArray(KmerCount7MTA.java:731)
        at kmer.KmerCount7MTA$CountThread.addRead_Advanced(KmerCount7MTA.java:705)
        at kmer.KmerCount7MTA$CountThread.count(KmerCount7MTA.java:502)
        at kmer.KmerCount7MTA$CountThread.run(KmerCount7MTA.java:461)

[Pierre Lindenbaum]

which version is it ?

[GenoMax]

Tagging: Brian Bushnell

@Brian validates BBMap only using Java 1.7 and above.

[smonsma]

I'm on 1.8:

java -version openjdk version "1.8.0_131 OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-0ubuntu1.16.04.2-b11 OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

The weird thing is, even just trying to make the histogram with k=43 fails, but the default (k=31) succeeds. I'm trying the normalization with default k now.

[smonsma]

Aggh. I moved this to a comment but lost the RAM question. Java is getting 371.5Gb, machine has 512 so I could conceivably give it more. Annnd, I've hit the post limit for noobs. Version 34.82 of bbtools.

[Brian Bushnell]

The memory is not a problem; BBNorm doesn't run out of memory. And by default it tries to use most, but not all, memory because our computers are configured to kill processes that use more than... ~85% of physical memory, for some reason. Some is reserved for caching, and some for the OS, so it's probably best to not try to use 100% of the memory in any case.

The latest version of BBTools is 37.32, so I suggest you try that and see if the error persists.

[Pierre Lindenbaum]

I was asking for the version of bbnorm