Question: How to parallelize a rule in Snakemake ?
1
gravatar for darbinator
9 months ago by
darbinator190
darbinator190 wrote:

Hi !

I started using snakemake to replace my bash scripts in order to have a more "pretty" code, but I have some problems, especially to parallel my jobs.

I have this code:

FILES = [ os.path.basename(x) for x in glob.glob("Experience/*") ] 

SAMPLES = list(set([ "_".join(x.split("_")[:2]) for x in FILES]))

CONDITIONS = list(set(x.split("_")[0] for x in SAMPLES))


for path in DIRS:
    if not os.path.exists(path):
        os.mkdir(path)


rule all:
    input:
        expand('Trimming/{sample}_R1.trim.fastq', sample=SAMPLES)


rule trimming:
    input:
        adapters = ADAPTERS,
        r1 = 'Experience/{sample}_R1.fastq.gz',
        r2 = 'Experience/{sample}_R2.fastq.gz'

    output:
        r1 = 'Trimming/{sample}_R1.trim.fastq',
        r2 = 'Trimming/{sample}_R2.trim.fastq'

    message: ''' --- Trimming  --- '''

    shell: ' bbduk.sh in1="{input.r1}" in2="{input.r2}" out1="{output.r1}" out2="{output.r2}" \
        ref="{input.adapters}" minlen='+str(minlen)+' ktrim='+ktrim+' k='+str(k)+' qtrim='+qtrim+' trimq='+str(trimq)+' hdist='+str(hdist)+' tpe tbo '

I have 5 samples, having used the wildcard "sample", I was expecting that my 5 trimming start at the same time, but they start one after the other .. What's wrong with my code?

thank you in advance

rna-seq snakemake pipeline ngs • 668 views
ADD COMMENTlink written 9 months ago by darbinator190
3
gravatar for WouterDeCoster
9 months ago by
Belgium
WouterDeCoster42k wrote:

How did you start the pipeline? Did you use the -j option of snakemake to allow multiple jobs?

ADD COMMENTlink written 9 months ago by WouterDeCoster42k

effectively, when I launch my script with this command " snakemake -j ", The 5 jobs start at the same time (I used to simply just execute "snakemake" before), so thank's you !

ADD REPLYlink written 9 months ago by darbinator190

Glad it was that easy :) I have moved my comment to an answer so you can mark it as accepted, and as such marking this thread as solved.

ADD REPLYlink written 9 months ago by WouterDeCoster42k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1907 users visited in the last hour