Entering edit mode
4.5 years ago
loum
•
0
Versions: nextflow 20.10.0 hecda079_0 bioconda, openjdk 8.0.152 h7b6447c_3, picard 2.25.0
I saw this post Picard MarkDuplicates fatal error and implemented use_jdk_deflater=true and use_jdk_inflater=true
#!/bin/bash
picard -Xmx120g MarkDuplicates ASSUME_SORT_ORDER=coordinate INPUT="127810_165_polyG_dimers_threeprime_quality_bwa-mem_sorted_downsample.bam" OUTPUT="127810_165_polyG_dimers_threeprime_quality_bwa-mem_sorted_downsample_markdups.bam" METRICS_FILE="127810_165_polyG_dimers_threeprime_quality_bwa-mem_sorted_downsample_markdups-metrics.txt" USE_JDK_DEFLATER=true USE_JDK_INFLATER=true
But I get the following error output from hs_err_pid9651.log:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002aaaabf7adfe, pid=9651, tid=0x00002aaaaaaea700
#
# JRE version: OpenJDK Runtime Environment (8.0_152-b12) (build 1.8.0_152-release-1056-b12)
# Java VM: OpenJDK 64-Bit Server VM (25.152-b12 mixed mode linux-amd64 )
# Problematic frame:
# V [libjvm.so+0x8a6dfe] SR_handler(int, siginfo*, ucontext*)+0x3e
#
# Core dump written. Default location: /.../core or core.9651
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread is native thread
siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000000000110
Registers:
RAX=0x0000000000000000, RBX=0x00002aaaaaaea670, RCX=0xffffffffffffffff, RDX=0x00002aaaaaaeaa20
RSP=0x00007fffffff3240, RBP=0x00007fffffff32f0, RSI=0x00007fffffff3430, RDI=0x0000000000000001
R8 =0x0000000000000000, R9 =0x00002aaaaabec700, R10=0x0000000000000000, R11=0x0000000000000246
R12=0x00002aaaaabecd28, R13=0x0000000000000002, R14=0x00007fffffff3430, R15=0x00007fffffff3300
RIP=0x00002aaaabf7adfe, EFLAGS=0x0000000000010246, CSGSFS=0x0000000000000033, ERR=0x0000000000000004
TRAPNO=0x000000000000000e
Top of Stack: (sp=0x00007fffffff3240)
0x00007fffffff3240: 0000000000000004 0000000000000008
0x00007fffffff3250: 000000007972d654 00002aaaaaab4d6b
0x00007fffffff3260: 00007fffffff3380 00007fffffff32a0
0x00007fffffff3270: 00002aaaab30929c 00002aaaab312e60
0x00007fffffff3280: 00007fffffff33b0 00007fffffff33a0
0x00007fffffff3290: 0000000000000014 00002aaaaaab4d6b
0x00007fffffff32a0: 0000000000000000 00002aaaaaae9698
0x00007fffffff32b0: 00002aaaaaae8ed8 00002aaaaacd1f87
0x00007fffffff32c0: 00002aaaab316fd0 00002aaaaabec700
0x00007fffffff32d0: 00002aaaaabecd28 00007fffffff3950
0x00007fffffff32e0: 0000000000100000 00007fffffff38b0
0x00007fffffff32f0: 00002aaaaaaea700 00002aaaaacde5f0
0x00007fffffff3300: 0000000000000001 0000000000000000
0x00007fffffff3310: 0000000000000000 00002aaa00000002
0x00007fffffff3320: 0000000000000000 0000000000000000
0x00007fffffff3330: 00002aaaaabec700 0000000000000000
0x00007fffffff3340: 0000000000000246 00002aaaaabecd28
0x00007fffffff3350: 00007fffffff3950 0000000000100000
0x00007fffffff3360: 00007fffffff38b0 00002aaaaabec9d0
0x00007fffffff3370: 0000000000000000 00002aaaaaaea700
0x00007fffffff3380: 00002aaaaabec700 00000000000025bf
0x00007fffffff3390: 00000000000000ca ffffffffffffffff
0x00007fffffff33a0: 00007fffffff38b0 00002aaaaacd7fd5
0x00007fffffff33b0: 0000000000000246 0000000000000033
0x00007fffffff33c0: 0000000000000000 0000000000000000
0x00007fffffff33d0: 0000000000000000 0000000000000000
0x00007fffffff33e0: 00007fffffff34c0 00002aaaaaae97a0
0x00007fffffff33f0: 00002aaa00000005 0000000000000000
0x00007fffffff3400: 0000000000000001 00002aaaaaae8508
0x00007fffffff3410: 000000001c9394d6 00002aaaaaee6018
0x00007fffffff3420: 0000000000000000 0000000000000000
0x00007fffffff3430: 000000000000000c 00002aaa00000000
I've also tried these versions: nextflow 20.10.0 hecda079_0 bioconda, openjdk 8.0.152 h46b5887_1, picard 2.18.14 0 bioconda
I did this (tried with older versions of openjdk and picard) because it has worked in the past with snakemake.
The command:
#!/bin/bash
picard -Xmx120G MarkDuplicates ASSUME_SORT_ORDER=coordinate INPUT="127810_165_polyG_dimers_threeprime_quality_bwa-mem_sorted_downsample.bam" OUTPUT="127810_165_polyG_dimers_threeprime_quality_bwa-mem_sorted_downsample_markdups.bam" METRICS_FILE="127810_165_polyG_dimers_threeprime_quality_bwa-mem_sorted_downsample_markdups-metrics.txt"
Here is part of the output in hs_err_pid9096.log:
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002aaaabf7adfe, pid=9096, tid=0x00002aaaaaaea200
#
# JRE version: OpenJDK Runtime Environment (8.0_152-b12) (build 1.8.0_152-release-1056-b12)
# Java VM: OpenJDK 64-Bit Server VM (25.152-b12 mixed mode linux-amd64 )
# Problematic frame:
# V [libjvm.so+0x8a6dfe] SR_handler(int, siginfo*, ucontext*)+0x3e
#
# Core dump written. Default location: /.../core or core.9096
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread is native thread
siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000000000110
Registers:
RAX=0x0000000000000000, RBX=0x00002aaaaaaea170, RCX=0xffffffffffffffff, RDX=0x00002aaaaaaea520
RSP=0x00007fffffff3a40, RBP=0x00007fffffff3af0, RSI=0x00007fffffff3c30, RDI=0x0000000000000001
R8 =0x0000000000000000, R9 =0x00002aaaaabec700, R10=0x0000000000000000, R11=0x0000000000000246
R12=0x00002aaaaabecd28, R13=0x0000000000000002, R14=0x00007fffffff3c30, R15=0x00007fffffff3b00
RIP=0x00002aaaabf7adfe, EFLAGS=0x0000000000010246, CSGSFS=0x0000000000000033, ERR=0x0000000000000004
TRAPNO=0x000000000000000e
Top of Stack: (sp=0x00007fffffff3a40)
0x00007fffffff3a40: 0000000000000004 0000000000000008
0x00007fffffff3a50: 000000007972d654 00002aaaaaab4d6b
0x00007fffffff3a60: 0000008000000000 00007fffffff3aa0
0x00007fffffff3a70: 00002aaaab30929c 00002aaaab312e60
0x00007fffffff3a80: 00007fffffff3bb0 00007fffffff3ba0
0x00007fffffff3a90: 0000000000000014 00002aaaaaab4d6b
0x00007fffffff3aa0: 0000000000000000 00002aaaaaae9190
0x00007fffffff3ab0: 00002aaaaaae89d0 00002aaaaacd1f87
0x00007fffffff3ac0: 00002aaaab316fd0 00002aaaaabec700
0x00007fffffff3ad0: 00002aaaaabecd28 00007fffffff4130
0x00007fffffff3ae0: 0000000000100000 00007fffffff4090
0x00007fffffff3af0: 00002aaaaaaea200 00002aaaaacde5f0
0x00007fffffff3b00: 0000000000000001 0000000000000000
0x00007fffffff3b10: 0000000000000000 00002aaa00000002
0x00007fffffff3b20: 0000000000000000 0000000000000000
0x00007fffffff3b30: 00002aaaaabec700 0000000000000000
0x00007fffffff3b40: 0000000000000246 00002aaaaabecd28
0x00007fffffff3b50: 00007fffffff4130 0000000000100000
0x00007fffffff3b60: 00007fffffff4090 00002aaaaabec9d0
0x00007fffffff3b70: 0000000000000000 00002aaaaaaea200
0x00007fffffff3b80: 00002aaaaabec700 00000000000023a1
0x00007fffffff3b90: 00000000000000ca ffffffffffffffff
0x00007fffffff3ba0: 00007fffffff4090 00002aaaaacd7fd5
0x00007fffffff3bb0: 0000000000000246 0000000000000033
0x00007fffffff3bc0: 0000000000000000 0000000000000000
0x00007fffffff3bd0: 0000000000000000 0000000000000000
0x00007fffffff3be0: 00007fffffff3cc0 00002aaaaaae9298
0x00007fffffff3bf0: 00002aaa00000005 0000000000000000
0x00007fffffff3c00: 0000000000000001 00002aaaaaae8000
0x00007fffffff3c10: 00000000ff878ec2 00002aaaaaee6018
0x00007fffffff3c20: 0000000000000000 0000000000000000
0x00007fffffff3c30: 000000000000000c 00002aaa00000000
Both error logs are truncated but can provide the full log.
Within nextflow.config, the -V propogates the conda environment to any new qsub job and wgs.config points to config files including one for picard MarkDuplicates (see below):
run_process_wgs {
includeConfig 'share.config'
configdir="config"
includeConfig 'wgs.config'
process.executor = 'sge'
process.queue = 'all.q'
process.penv = 'smp'
process.clusterOptions = { "-l h_vmem=${task.ext.vmem} -S /bin/bash -P project_grp -V" }
}
picard.config:
process {
withName: picard_mark_dups {
memory = '120 GB'
cpus = 1
ext {
command ={"$workflow.projectDir/config/mark_dups/picard/picard.sh"}
vmem = '130G'
}
}
}
I've tried a small java heap and increasing the java heap to 200G but I still get an error.
I've tried with a smaller sized data set and it has worked on one node and on the cluster.
Both errors given in this post have exit code 140.
Any help would be appreciated! Please let me know if you need more information.
what is your OS ?
oh I see
you should tell about your problem here: https://github.com/broadinstitute/picard/issues/1329