deeparg-takes-too-many-threads

Question

[shevch2009]

Hello, all!

I was trying to use deeparg for raw shotgun reads, but when it goes into Vsearch it tends to take too many threads, that we are allowed to use on the server. There are no options to put number of threads directly, it's says - unrecognized argument. Because it's in conda env, I have tried to use, -

OMP_THREAD_LIMIT=20
export OMP_THREAD_LIMIT

export OMP_NUM_THREADS=20

But, it's still running with much more threads that I can use.

I would appreciate any suggestions.

Best, Alla

[Dave Carlson]

Note that vsearch uses pthreads, not OpenMP, for multithreading. Therefore setting OpenMP environment variables (e.g., OMP_NUM_THREADS) will have no effect.

[GenoMax]

Related past thread for reference: concoct in metawrap binning module - use all available threads instead of requested , in case you know about the programs mentioned.

[Dave Carlson]

I'm not particularly familiar with Concoct, but from glancing at the code, it does appear to use OpenMP, so I would expect that setting OMP_NUM_THREADS should work to control the number of threads produced in that case.

[GenoMax]

If the program is not respecting limits you are setting then you may need to dive into code to see why that is happening. It may or may not be fixable. If you are like most other users, a non-programmer, then there is not much you can do but look alternatives that may work.

Obviously you are not running this under a job scheduler, since it would not allow programs to use more resources than assigned.

[shevch2009]

Thanks, I will look for another tools. Yes, I am running it withing a screen session.

[GenoMax]

I am running it withing a screen session

If you are using screen then you may need to set the variables in a different way. Try to see which of the two works for you

https://stackoverflow.com/questions/13974069/setting-environment-variables-in-screenrc
https://stackoverflow.com/questions/1318210/gnu-screen-environment-variables