Question: How to parallelize a rule in Snakemake ?
gravatar for darbinator
4 weeks ago by
darbinator40 wrote:

Hi !

I started using snakemake to replace my bash scripts in order to have a more "pretty" code, but I have some problems, especially to parallel my jobs.

I have this code:

FILES = [ os.path.basename(x) for x in glob.glob("Experience/*") ] 

SAMPLES = list(set([ "_".join(x.split("_")[:2]) for x in FILES]))

CONDITIONS = list(set(x.split("_")[0] for x in SAMPLES))

for path in DIRS:
    if not os.path.exists(path):

rule all:
        expand('Trimming/{sample}_R1.trim.fastq', sample=SAMPLES)

rule trimming:
        adapters = ADAPTERS,
        r1 = 'Experience/{sample}_R1.fastq.gz',
        r2 = 'Experience/{sample}_R2.fastq.gz'

        r1 = 'Trimming/{sample}_R1.trim.fastq',
        r2 = 'Trimming/{sample}_R2.trim.fastq'

    message: ''' --- Trimming  --- '''

    shell: ' in1="{input.r1}" in2="{input.r2}" out1="{output.r1}" out2="{output.r2}" \
        ref="{input.adapters}" minlen='+str(minlen)+' ktrim='+ktrim+' k='+str(k)+' qtrim='+qtrim+' trimq='+str(trimq)+' hdist='+str(hdist)+' tpe tbo '

I have 5 samples, having used the wildcard "sample", I was expecting that my 5 trimming start at the same time, but they start one after the other .. What's wrong with my code?

thank you in advance

rna-seq snakemake pipeline ngs • 115 views
ADD COMMENTlink written 4 weeks ago by darbinator40
gravatar for WouterDeCoster
4 weeks ago by
WouterDeCoster37k wrote:

How did you start the pipeline? Did you use the -j option of snakemake to allow multiple jobs?

ADD COMMENTlink written 4 weeks ago by WouterDeCoster37k

effectively, when I launch my script with this command " snakemake -j ", The 5 jobs start at the same time (I used to simply just execute "snakemake" before), so thank's you !

ADD REPLYlink written 4 weeks ago by darbinator40

Glad it was that easy :) I have moved my comment to an answer so you can mark it as accepted, and as such marking this thread as solved.

ADD REPLYlink written 4 weeks ago by WouterDeCoster37k
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1200 users visited in the last hour