Samtools: "Too Many Open Files"
2
4
Entering edit mode
13.0 years ago
Zhshqzyc ▴ 520

Hello, I used samtools to sort a 210GB bam file. The command was:

samtools sort -on in.bam out.prefix

About 1058 .bam files were created. Each of them has about 200,000kb size. The names are

out.prefix.0000.bam
out.prefix.0001.bam
out.prefix.0002.bam
....
out.prefix.1057.bam

However after several days running, there was an error finally.

[bam_sort_core] merging from 1249 files...
open: Too many open files
[bam_merge_core] fail to open file out.prefix.1020.bam

Thanks for advice.

samtools • 13k views
ADD COMMENT
1
Entering edit mode

Hi zhshqzyc! You have asked 15 questions so far and not accepted a single answer. Please click on the outline of the checkmark next to it if it solves your problem.

ADD REPLY
0
Entering edit mode

Sorry about it. I just realized there is such a function existing.

ADD REPLY
9
Entering edit mode
13.0 years ago

It is common to have a soft limit of 1024 open files by default. See what

ulimit -n

tells you. If it is 1024 then it is likely that your system allows more files to be open only that the ulimit imposes the per-process limit of 1024. See the manual page for ulimit on how to change this.

ADD COMMENT
6
Entering edit mode
13.0 years ago
brentp 24k

You can increase the argument for maxMem via -m. This will reduce the number of temporary files.

The default is 500000000 so try 10 times that value.

ADD COMMENT

Login before adding your answer.

Traffic: 3434 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6