run "sra-human-scrubber" with SLURM
0
0
Entering edit mode
9 weeks ago
emmanouil.a ▴ 110

Hi, about sra-human-scrubber:

if I run in a terminal

./scrub.sh test


it works and I have as output

2022-07-30 15:55:18     aligns_to version 0.60
2022-07-30 15:55:18     hardware threads: 80, omp threads: 80
2022-07-30 15:55:19     loading time (sec) 0
2022-07-30 15:55:19     /tmp/tmp.q0E2fhyiOM/temp.fasta
2022-07-30 15:55:19     100% processed
2022-07-30 15:55:19     total spot count: 2
2022-07-30 15:55:19     total read count: 2
2022-07-30 15:55:19     total time (sec) 0


but when I run the same in a SLURM submitted job

#!/usr/bin/env bash
#SBATCH --cpus-per-task=6
#SBATCH --mem=25G
#SBATCH --time=02:00:00

./scrub.sh test


I have the following output

2022-07-30 15:55:35     aligns_to version 0.60
2022-07-30 15:55:35     hardware threads: 80, omp threads: 6
2022-07-30 15:55:35     loading time (sec) 0
2022-07-30 15:55:35     /tmp/tmp.7PZUfM7UT2/temp.fasta
2022-07-30 15:55:35     0% processed
2022-07-30 15:55:35     100% processed
2022-07-30 15:55:35     total spot count: 0
2022-07-30 15:55:35     total read count: 0
2022-07-30 15:55:35     total time (sec) 0
0  spot(s) removed.


I do not have any error. How can I make it works?

slurm sra-human-scrubber • 468 views
ADD COMMENT
0
Entering edit mode

I am going to guess that running test copies some file from the software bundle and then runs it. Depending on how your slurm cluster is setup the folder with the software may not be reachable from worker node where your job runs.

If you provide a different test file that you know has data then does scrub.sh work under SLURM. If it is does then above is your answer.

ADD REPLY
0
Entering edit mode

Hi, here testing the code using real samples (the tool's files have 770 permissions):

1) run directly in the terminal: **OK**
2) create a file locally "touch", add the code and run through it in the terminal: **OK**
3) create a file locally "touch", add the code and SLURM options, submit to SLURM: ***problem as in my original code here post***
4) open a SLURM interactive job and run the script as in 1) and 2): **OK**
5) run directly in the terminal many fastq files with a loop "for": **OK**
5) run directly in the terminal many fastq files with a loop "while": **problem as in my original code here post***

ADD REPLY
0
Entering edit mode

If you have SLURM on your cluster you should be using that no matter. Usefor or while to submit SLURM jobs but not for actually running analyses. Error with number 6 may simply be some logic issue with your while statement.

Thank you for trying various combinations. Number 3 above tells me that your cluster is setup in some specific way and worker nodes are still not able to reach the software folder. It may be best to contact your local system admins with the above set of data.

ADD REPLY
0
Entering edit mode

Thank you GenoMax.The Admin is on holidays and I'm at junior level for those thinks. I will try with "GNU parallel" but I will expect problems with the multithreading. Seems that the code "align_to" is using multithreading (whole system cpus without limitations) and it is out of my control.

ADD REPLY

Login before adding your answer.

Traffic: 1111 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6