Question: BBMap genome indexing - java issue?
0
gravatar for CAnna
10 months ago by
CAnna10
CAnna10 wrote:

Hi,

I want to remove human contaminants from gut metagenomes. For this I am using BBMap. The first step is to index the contaminant genome (human genome). I am following the exact procedure as described here, which I cross validated by finding the same commands described on other pages (biostars, github).

My command:

bbmap.sh -Xmx24G ref=hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz

is simply not working. My script outputs an output and error files. The output file is empty, the error files simply writes the command

java -Djava.library.path=/exports/csce/eddie/biology/groups/mcnally/camille/programs/bbmap/jni/ -ea -Xmx24G -cp /exports/csce/eddie/biology/groups/mcnally/camille/programs/bbmap/current/ align2.BBMap build=1 overwrite=true fastareadlen=500 -Xmx24G ref=hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz

and I have no other output.

I can't figure out the problem. Am I using the command wrong? Or is it an issue with my java? I am using my university cluster.

Thanks, Camille

bbmap java indexing • 513 views
ADD COMMENTlink modified 10 months ago • written 10 months ago by CAnna10

If you are using a cluster are you asking for a corresponding amount of RAM from the job scheduler? Are you using Java 7/8? You should also look at the log files of your job scheduler and post the errors present here.

ADD REPLYlink modified 10 months ago • written 10 months ago by genomax62k

Thanks for your reply

Yes I am specifying -l h_vmem=24G on the job scheduler

I am using Java 8 (java/jdk/1.8.0)

And the log file of the job scheduler is here (if this is what you meant?)

qname        eddie
hostname     node2c03.ecdf.ed.ac.uk
group        eddie_users
owner        s1687811
project      uoe_baseline
department   defaultdepartment
jobname      index_bbmap
jobnumber    1976327
taskid       undefined
account      sge
priority     0
qsub_time    Sun Apr  8 22:16:04 2018
start_time   Sun Apr  8 22:16:13 2018
end_time     Sun Apr  8 22:16:13 2018
granted_pe   NONE
slots        1
failed       0
exit_status  1
ru_wallclock 0
ru_utime     0.010
ru_stime     0.007
ru_maxrss    6128
ru_ixrss     0
ru_ismrss    0
ru_idrss     0
ru_isrss     0
ru_minflt    4243
ru_majflt    0
ru_nswap     0
ru_inblock   0
ru_oublock   72
ru_msgsnd    0
ru_msgrcv    0
ru_nsignals  0
ru_nvcsw     50
ru_nivcsw    67
cpu          0.017
mem          0.000
io           0.000
iow          0.000
maxvmem      0.000
arid         undefined
ADD REPLYlink modified 10 months ago • written 10 months ago by CAnna10

That is not telling us much.

I am running as indexing operation with the same file to test and so far no problem. I am using 30g of RAM. Will update later as to what happens.

ADD REPLYlink written 10 months ago by genomax62k

Ha, this is tricky, the thing is that I do not have any output/error to suggest what is wrong. I don't think it is a memory issue since maxvmem if I recall says how much memory has been used at max and this says 0. Looks like it did not run at all?

ADD REPLYlink written 10 months ago by CAnna10

You could start running it outside the job scheduler to make sure bbmap is working fine (be ready to kill the job). I was able to complete the indexing without any problems. I did use 30g of RAM.

ADD REPLYlink modified 10 months ago • written 10 months ago by genomax62k
1
gravatar for CAnna
10 months ago by
CAnna10
CAnna10 wrote:

Arg, I figured it out. For the sake of potential beginners having this kind of issue, this was related to my shell script.

If you do specify a value for -Xmx it needs to be less than the amount of virtual memory that you reserve for your job. In my case I had reserved 30GB of vmem, so should set -Xmx to something like:

-Xmx15G

as an example.

However, all this is moot if you are running Java 1.8. You shouldn't specify a value for -Xmx for this newer version of Java. You should only specify a value for this parameter if you are running Java 1.7 or lower.

So, in summary, re-running your job with this line with Java 1.8

bbmap.sh ref=hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz

or with Java 1.7

bbmap.sh -Xmx15G ref=hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz

You might have to experiment with the value of -Xmx depending on the genome you are eindexing but the key thing is it must be less than the amount of virtual memory you reserve for your job.

ADD COMMENTlink modified 10 months ago by genomax62k • written 10 months ago by CAnna10

While I don't fully understand this explanation (never had this issue with Java 7/8) as long as it worked for you that is good. This may be a local configuration specific thing. Thank you for coming back and posting a resolution.

ADD REPLYlink written 10 months ago by genomax62k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1135 users visited in the last hour