I’m moving from a HPC that had slurm for job scheduling to one with no job management program. Is there a program that can sorta manage lots of parallel jobs kinda like personal slurm? I’m using screen but it’s kinda annoying, I’d like to start a lot of jobs and log off and come back later.
Depending on how much effort you want to invest, you could run your program(s) via snakemake and let snakemake handle parallelism and job dependencies. There will be a learning curve but in my opionion is totally worth it.
xargs (available on *nix systems) you can run programs in parallel. For example, run the list of bash scripts captured by
ls, run up to 8 in parallel:
ls job_number.*.sh | xargs -P 8 -n 1 bash
parallel is an alternative to xargs, arguably more powerful than