multiIntersectBed Error: The requested bed file could not be opened. Exiting!

Question

[yren0429]

Hi fellow bioinformaticians,

I am trying to use multiIntersectBed to find intersections of genomic regions (copy number variations) in ~1400 samples. For each sample, I have a sorted bed file.

When I used multiIntersectBed on 5 bed files as a test, it worked perfectly. But when I used it on all ~1400 samples, I received the following error: "Error: The requested bed file (file1.bed) could not be opened. Exiting!"

However, this bed file runs fine if I only input a few bed files instead of 1400 . Therefore it seems to me like a problem regarding the number of files I can open at a time.

Does anyone know if there is a limit in the number of bed files that multiIntersectBed accepts? If so, is there a way to work around it?

Thank you very much for your help!

[finswimmer]

I guess this more a limitation by the OS.

On many linux distribution the maximum number of open files is 1024. You can find it out on your system with

$ ulimit -n

There seems to be many ways to increase the number if necessary, e.g. like here

fin swimmer

[Ram]

Do you mean the multiIntersectBed in bedtools 2.14.1? Can you give us the exact command you're using as well the bedtools version please?

Answer 1

[yren0429]

Great thanks to Ram and finswimmer for offering your help! Turns out it was indeed a problem with the max number of open files. The soft limit on my linux system is 1024, but the hard limit is 4096. I was able to increase my soft limit to up to 4096, and multiIntersectBed ran perfectly.

Below are codes to increase the soft limit for people who need it:

## view original soft limit
ulimit -Sn

## increase soft limit to 4096
ulimit -n 4096

##check soft limit again, now it should be 4096
ulimit -Sn

Many thanks again!