I know the computations by breaking the files and parallelize it in SGE or SLURM. I want to use whole computation power (20 node , 80 cores) for a single job without breaking it. If I just submit it (a single perl script for large computation by AutoFact program) from master node it is occupying the cluster but not computing with 20 node and 80 cores.
You'd have to rewrite your perl script to use MPI so that you can dole out computations between the nodes. Just running a program on a cluster doesn't make it parallel, you have to design it to use the resources.
Looking at the AutoFACT scripts, it's not clear that MPI will help. Looking over the methodology, it looks like it just does a bunch of searches to various databases, correct? If it's not a computationally intensive process for AutoFACT, that suggests a parallelization approach, where each of various searches can be set on their own node. Take a look at the Schedule::SGE module and consider modifying the AutoFACT code to support scheduling jobs, where jobs are database searches.
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
version 2.3.6
Hi,
I have a feeling more details (specifics on what you're trying to accomplish, what the perl script does, etc) will help you get an answer that is best suited to your scenario. The question in its current form is pretty vague and offers little detail on your *exact* current procedure. Pl help us help you better :)
"for a single job without breaking it" seems incompatible with "computing with 20 node and 80 cores."