Core utilization with Galaxy
0
1
Entering edit mode
7.5 years ago
anorman07 ▴ 10

Hi

I'm a complete Galaxy newbie, and am trying to configure a 16-core workstation to run, with an emphasis on speed-per-job. In other words, I'd like to dedicate as many cores to a given job as possible, and will run jobs sequentially.

This example job_conf.xml file was very well laid out and explained (from https://biostar.usegalaxy.org/p/10158/#23245), and I've attempted to modify it to run 14 cores on the listed tools. However, it's not working, and it looks like only one core is working when I run Cufflinks (for example). Any idea what I'm doing wrong? Are there any other configuration files that I need to tweak? I have a feeling I'm doing some that is obviously wrong, and I apologize if I am.

<job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="2"/> <plugin id="multilocal" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="1"/> </plugins>

<handlers> <handler id="main"/> </handlers>

<destinations default="local"> <destination id="local" runner="local"/> <destination id="multicore14" runner="multilocal"> <param id="local_slots">14</param> </destination> </destinations>

<tools> <tool id="cuffdiff" destination="multicore14"/> <tool id="cufflinks" destination="multicore14"/> <tool id="cuffmerge" destination="multicore14"/> <tool id="fastq_groomer" destination="multicore14"/> <tool id="fastqc" destination="multicore14"/> <tool id="rgrnastar" destination="multicore14"/> <tool id="trimmomatic" destination="multicore14"/> </tools>

</job_conf>

Galaxy multicore threads • 2.0k views
ADD COMMENT
1
Entering edit mode

You should post this over at Galaxy Biostars.

Not every program is able to use multiple cores so don't think you would be able to accelerate every program by making it use multiple cores.

ADD REPLY
0
Entering edit mode

genomax is right, to use more threads the feature has to be supported by both the software and the Galaxy tool. Galaxy tool authors specify it e.g. as --num-threads="\${GALAXY_SLOTS:-4}" in their tool wrapper. Only if both these conditions are fulfilled you can use the job_conf and set up a multi-thread destination for such tools.

check out https://galaxyproject.org/admin/config/galaxy_slots/

ADD REPLY

Login before adding your answer.

Traffic: 1704 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6