Question: Multi Thread Bedtools
1
gravatar for Nicolas Rosewick
7.1 years ago by
Belgium, Brussels
Nicolas Rosewick7.2k wrote:

Hi,

Is there a multi thread version of bedtools ? or is this feature in development ?

Thanks,

N.

bedtools parallel • 6.3k views
ADD COMMENTlink modified 4.5 years ago by Biostar ♦♦ 20 • written 7.1 years ago by Nicolas Rosewick7.2k
1

Perhaps GNU parallel and bedtools?

ADD REPLYlink written 7.1 years ago by Martin A Hansen3.0k

I think I will write a little perl script to divide the gff file into multiple gff files (one gff file per feature) and execute multiple threads (one thread per gff file) in parallel. Sounds good no ?

ADD REPLYlink written 7.1 years ago by Nicolas Rosewick7.2k

You might consider pybedtools instead.

ADD REPLYlink written 7.1 years ago by Aaronquinlan10k
6
gravatar for Aaronquinlan
7.1 years ago by
Aaronquinlan10k
United States
Aaronquinlan10k wrote:

Bedtools is currently not multi-threaded and won't be for the foreseeable future. Multi-threading typically yields performance gains when the relevant data structures are stored in memory, so that the gains of additional threads are not eliminated by I/O constraints. In contrast, most of the bedtools algorithms read intervals from the driver (A) file line by line from disk in order to minimize memory consumption (e.g., loading a BAM file into memory is not advisable). This restriction, combined with the fact that, for many cases (e.g. sorted BAM), we really need the output to preserve the order of the original input. This poses additional restrictions on multi-threading.

That said, I have been working on scalable new algorithms for both distributed computing environments (clusters) and shared memory systems such as GPUs.

If anyone has the time and ability to effectively multi-thread the existing tools, I would be more than happy to collaborate!

ADD COMMENTlink written 7.1 years ago by Aaronquinlan10k
2
gravatar for Ryan Dale
7.1 years ago by
Ryan Dale4.8k
Bethesda, MD
Ryan Dale4.8k wrote:

The third pybedtools example shows how to do this with BEDTools in Python, but it's limited to embarrassingly parallel problems.

ADD COMMENTlink written 7.1 years ago by Ryan Dale4.8k
1
gravatar for Damian Kao
7.1 years ago by
Damian Kao15k
USA
Damian Kao15k wrote:

Bedtools Future section

This was last updated more than a year ago though.

ADD COMMENTlink written 7.1 years ago by Damian Kao15k

ouch. I think I should rename that page as "Past work"

ADD REPLYlink written 7.1 years ago by Aaronquinlan10k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1288 users visited in the last hour