How to parallelize a rule in Snakemake ?

Question

[vin.darb]

Hi !

I started using snakemake to replace my bash scripts in order to have a more "pretty" code, but I have some problems, especially to parallel my jobs.

I have this code:

FILES = [ os.path.basename(x) for x in glob.glob("Experience/*") ] 

SAMPLES = list(set([ "_".join(x.split("_")[:2]) for x in FILES]))

CONDITIONS = list(set(x.split("_")[0] for x in SAMPLES))


for path in DIRS:
    if not os.path.exists(path):
        os.mkdir(path)


rule all:
    input:
        expand('Trimming/{sample}_R1.trim.fastq', sample=SAMPLES)


rule trimming:
    input:
        adapters = ADAPTERS,
        r1 = 'Experience/{sample}_R1.fastq.gz',
        r2 = 'Experience/{sample}_R2.fastq.gz'

    output:
        r1 = 'Trimming/{sample}_R1.trim.fastq',
        r2 = 'Trimming/{sample}_R2.trim.fastq'

    message: ''' --- Trimming  --- '''

    shell: ' bbduk.sh in1="{input.r1}" in2="{input.r2}" out1="{output.r1}" out2="{output.r2}" \
        ref="{input.adapters}" minlen='+str(minlen)+' ktrim='+ktrim+' k='+str(k)+' qtrim='+qtrim+' trimq='+str(trimq)+' hdist='+str(hdist)+' tpe tbo '

I have 5 samples, having used the wildcard "sample", I was expecting that my 5 trimming start at the same time, but they start one after the other .. What's wrong with my code?

thank you in advance

Answer 1

[WouterDeCoster]

How did you start the pipeline? Did you use the -j option of snakemake to allow multiple jobs?

[vin.darb]

effectively, when I launch my script with this command " snakemake -j ", The 5 jobs start at the same time (I used to simply just execute "snakemake" before), so thank's you !

[WouterDeCoster]

Glad it was that easy :) I have moved my comment to an answer so you can mark it as accepted, and as such marking this thread as solved.