I have a question about how to run Snakemake tool properly on the HPC (cluster) and I understand that there are at least three ways of doing this below:
1) The following is the content of a test.sh shell script.
#!/bin/bash #$ -cwd #$ -V #$ -l h_rt=48:00:00 #$ -l nodes=10,ppn=1 snakemake -p --cores 10 --snakefile test.snakemake
The way of running is to run "
qsub test.sh" to submit this job to the compute nodes of HPC.
2) Use the --cluster argument suggested by the Snakemake manual (https://snakemake.readthedocs.io/en/v5.1.4/executable.html) as below:
snakemake --cluster qsub -j 32 --snakefile test.snakemake
3) Use the --profile argument suggested by the Snakemake manual (https://snakemake.readthedocs.io/en/v5.1.4/executable.html) as below:
snakemake --profile myprofile --snakefile test.snakemake
I was wondering if the three methods can lead to the same effects. If not, what would be the difference? Although I was told that the first approach is not going to make use of the multiple nodes of HPC, I don't see that it is true as in practice it seems to have made use of the 10 nodes of HPC.