Question: How to parallelize a rule in Snakemake ?
1
gravatar for darbinator
4 weeks ago by
darbinator40
darbinator40 wrote:

Hi !

I started using snakemake to replace my bash scripts in order to have a more "pretty" code, but I have some problems, especially to parallel my jobs.

I have this code:

FILES = [ os.path.basename(x) for x in glob.glob("Experience/*") ] 

SAMPLES = list(set([ "_".join(x.split("_")[:2]) for x in FILES]))

CONDITIONS = list(set(x.split("_")[0] for x in SAMPLES))


for path in DIRS:
    if not os.path.exists(path):
        os.mkdir(path)


rule all:
    input:
        expand('Trimming/{sample}_R1.trim.fastq', sample=SAMPLES)


rule trimming:
    input:
        adapters = ADAPTERS,
        r1 = 'Experience/{sample}_R1.fastq.gz',
        r2 = 'Experience/{sample}_R2.fastq.gz'

    output:
        r1 = 'Trimming/{sample}_R1.trim.fastq',
        r2 = 'Trimming/{sample}_R2.trim.fastq'

    message: ''' --- Trimming  --- '''

    shell: ' bbduk.sh in1="{input.r1}" in2="{input.r2}" out1="{output.r1}" out2="{output.r2}" \
        ref="{input.adapters}" minlen='+str(minlen)+' ktrim='+ktrim+' k='+str(k)+' qtrim='+qtrim+' trimq='+str(trimq)+' hdist='+str(hdist)+' tpe tbo '

I have 5 samples, having used the wildcard "sample", I was expecting that my 5 trimming start at the same time, but they start one after the other .. What's wrong with my code?

thank you in advance

rna-seq snakemake pipeline ngs • 115 views
ADD COMMENTlink written 4 weeks ago by darbinator40
3
gravatar for WouterDeCoster
4 weeks ago by
Belgium
WouterDeCoster37k wrote:

How did you start the pipeline? Did you use the -j option of snakemake to allow multiple jobs?

ADD COMMENTlink written 4 weeks ago by WouterDeCoster37k

effectively, when I launch my script with this command " snakemake -j ", The 5 jobs start at the same time (I used to simply just execute "snakemake" before), so thank's you !

ADD REPLYlink written 4 weeks ago by darbinator40

Glad it was that easy :) I have moved my comment to an answer so you can mark it as accepted, and as such marking this thread as solved.

ADD REPLYlink written 4 weeks ago by WouterDeCoster37k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1200 users visited in the last hour