Hey,
I have a problem with the picard tool: Mark Duplicates
To get an overview: I have a *.bam input file which is sorted. And I want to use the MarkDuplicates-tool from picard.jar. First I tried to run it with default. But I got an error, which was solved by adding "READ_NAME_REGEX=null \". Now, my command is:
java -jar /home/jules/programs/picard/build/libs/picard.jar MarkDuplicates \
I=CCO_bowtie2Mapping.sorted.bam \
O=CCO_picard_DM.bam \
M=CCO_marked_dup_metrics.txt \
READ_NAME_REGEX=null \
But I got a new error. Unfortunately, I cannot detect the real error in this bunch of notes the terminal is showing me.
[Mon Feb 20 13:53:03 CET 2017] picard.sam.markduplicates.MarkDuplicates SORTING_COLLECTION_SIZE_RATIO=0.1 INPUT=[CCO_bowtie2Mapping.sorted.bam] OUTPUT=CCO_picard_DM.bam METRICS_FILE=CCO_marked_dup_metrics.txt READ_NAME_REGEX=null TMP_DIR=[/home/jules/programs/picard/build/tmp/compileJava/emptySourcePathRef/working_tmp] MAX_SEQUENCES_FOR_DISK_READ_ENDS_MAP=50000 MAX_FILE_HANDLES_FOR_READ_ENDS_MAP=8000 REMOVE_SEQUENCING_DUPLICATES=false TAGGING_POLICY=DontTag REMOVE_DUPLICATES=false ASSUME_SORTED=false DUPLICATE_SCORING_STRATEGY=SUM_OF_BASE_QUALITIES PROGRAM_RECORD_ID=MarkDuplicates PROGRAM_GROUP_NAME=MarkDuplicates OPTICAL_DUPLICATE_PIXEL_DISTANCE=100 VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json
[Mon Feb 20 13:53:03 CET 2017] Executing as jules@jules-TERRA-PC on Linux 4.4.0-62-generic amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_111-b14; Picard version: 2.7.1-24-gc8bbc7a-SNAPSHOT
INFO 2017-02-20 13:53:03 MarkDuplicates Start of doWork freeMemory: 245340416; totalMemory: 251658240; maxMemory: 3726639104
INFO 2017-02-20 13:53:03 MarkDuplicates Reading input file and constructing read end information.
INFO 2017-02-20 13:53:03 MarkDuplicates Will retain up to 5733290 data points before spilling to disk.
[Mon Feb 20 13:53:08 CET 2017] picard.sam.markduplicates.MarkDuplicates done. Elapsed time: 0.09 minutes.
Runtime.totalMemory()=1812987904
To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
Exception in thread "main" htsjdk.samtools.SAMException: /home/jules/programs/picard/build/tmp/compileJava/emptySourcePathRef/working_tmp/CSPI.6417316537446830880.tmp/1567.tmpnot found
at htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.makeValue(FileAppendStreamLRUCache.java:63)
at htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.makeValue(FileAppendStreamLRUCache.java:49)
at htsjdk.samtools.util.ResourceLimitedMap.get(ResourceLimitedMap.java:76)
at htsjdk.samtools.CoordinateSortedPairInfoMap.getOutputStreamForSequence(CoordinateSortedPairInfoMap.java:180)
at htsjdk.samtools.CoordinateSortedPairInfoMap.ensureSequenceLoaded(CoordinateSortedPairInfoMap.java:102)
at htsjdk.samtools.CoordinateSortedPairInfoMap.remove(CoordinateSortedPairInfoMap.java:86)
at picard.sam.markduplicates.util.DiskBasedReadEndsForMarkDuplicatesMap.remove(DiskBasedReadEndsForMarkDuplicatesMap.java:61)
at picard.sam.markduplicates.MarkDuplicates.buildSortedReadEndLists(MarkDuplicates.java:471)
at picard.sam.markduplicates.MarkDuplicates.doWork(MarkDuplicates.java:222)
at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:208)
at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:95)
at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:105)
Caused by: java.io.FileNotFoundException: /home/jules/programs/picard/build/tmp/compileJava/emptySourcePathRef/working_tmp/CSPI.6417316537446830880.tmp/1567.tmp (Too many open files)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.makeValue(FileAppendStreamLRUCache.java:60)
... 11 more
I tried to increase the number of open folders, but it did not work with the unlimited number. Hopefully, some can help me. Thanks a lot. Julia