Question: GATB Multithread always one core
0
gravatar for lys8918
6 months ago by
lys89180
lys89180 wrote:

Hi,

I'm writing some codes using GATB. First, I write a sample code to test the multithread. But, why this code always run on one core? Any problems of my code?

The code from examples provided by GATB (multithreading1.cpp). I add some code in Functor to waste times.

 #include <gatb/gatb_core.hpp>
using namespace std;
/********************************************************************************/
/*                    Multithreaded iteration of an integer range               */
/********************************************************************************/
// We define a functor that will be cloned by the dispatcher
struct Functor { void operator() (int i)
{
    double s = 0;
    char str[100];
    sprintf(str, "%d.txt", i);
    FILE *fp = fopen(str, "w");
    for (int i = 0; i < 100000; i++) {
        s += i/10000.0;
        s *= 3.0;
        fprintf(fp, "%lf\n", s);
    }
    fclose(fp);
    cerr<<i<<"\n";
    // In this instruction block, we are executing in one of the nbCores threads
    // created by the dispatcher. Note that 'i' is one value of our range
}};
/********************************************************************************/
int main (int argc, char* argv[])
{
    // We get the number of cores to be used.  If we don't give any number,
    // we set to 0 which implies the usage of all available cores
    size_t nbCores = (argc >=2 ? atoi(argv[1]) : 0);
    // We create an iterator over an integer range
    Range<int>::Iterator it (1,1000);
    // We create a dispatcher configured for 'nbCores' cores.
    Dispatcher dispatcher (nbCores);
    // We dispatch the range iteration with the dispatcher.
    // This will create nbCores threads and each thread will be fed with
    // one value of the defined range
    // NOTE: we could also use lambda expression (easing the code readability)
    IDispatcher::Status status = dispatcher.iterate (it, Functor());
    // We dump some information about the dispatching
    cout << "nbCores=" << status.nbCores << "  time=" << status.time << endl;
    // IMPORTANT: usage of Dispatcher has sense only if the iterated items
    // can be processed independently from each other.
    // The point to understand with the Dispatcher is that it can
    // iterate any instance of Iterator class. If you have any set of items
    // that can be enumerated through an Iterator implementation, then you
    // can parallelize the iteration with a Dispatcher instance
}
gatb • 246 views
ADD COMMENTlink modified 5 months ago by Rayan Chikhi1.2k • written 6 months ago by lys89180

indeed, I can confirm the issue. For some reason the dispatcher only uses one core in that setting. Will see how to suggest fix.

ADD REPLYlink written 5 months ago by Rayan Chikhi1.2k
0
gravatar for Rayan Chikhi
5 months ago by
Rayan Chikhi1.2k
France, Lille, CNRS
Rayan Chikhi1.2k wrote:

Hi, We've found the issue here, thank to @grizk. Please change

 IDispatcher::Status status = dispatcher.iterate (it, Functor());

to

 IDispatcher::Status status = dispatcher.iterate (it, Functor(), 1);

This is because the third argument of iterate() is a batch size, i.e. a number of iterations that are batched into a single job in a thread. By default it is set to 1000, as the elements that are iterated on are typically kmers from a large set. Setting a large batch size avoids the overhead of switching threads for each k-mer. Here, by setting a batch size of 1, each Functor() is executed as soon as a thread is available.

ADD COMMENTlink modified 5 months ago • written 5 months ago by Rayan Chikhi1.2k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 706 users visited in the last hour