Is it possible to pass command line arguments to python script in snakemake?
1
1
Entering edit mode
2.9 years ago
blackadder ▴ 30

Hello all,

When I am executing my snakemake pipeline I am trying to pass command line arguments with the --config flag in a python script.

rule assembly:
input:
    "trim/{sample}_1.fastq.gz",
    "trim/{sample}_2.fastq.gz",
output:
    "assembly/{sample}.fa"
params:
    fastq1 = "trim/{sample}_1.fastq.gz",
    fastq2 = "trim/{sample}_2.fastq.gz",
conda: "environment.yaml"
script:
    "assembly.py"

The Python script

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--config', type=str, required=True)
args = parser.parse_args()

if args.config["assembly"] == "megahit":
    shell ("megahit .........")
elif config["assembly"] == "metaspades":
    shell ("metaspades.py ........")

I am executing the pipeline like this:

snakemake --cores 5 --config assembly="megahit"

or

snakemake --cores 5 --config assembly="metaspades" 

When I do that I am getting the following error:

error: the following arguments are required: --config

  1. Is there a way to make this work?
  2. Am I completely wrong by choosing this option? (I did because when I used the snakemake run directive I was getting the "conda environments are only allowed with shell script notebook or wrapper directives (not with run)" error.

Thanking you in advance!

snakemake python • 5.1k views
ADD COMMENT
0
Entering edit mode

i don't think you need to add the addparse stuff, just use a generic configfile with a default value and your inline arguments should work

ADD REPLY
0
Entering edit mode

So, you suggest instead of --config use the --configfile and specify the different params in this one?

ADD REPLY
0
Entering edit mode

all this is unnecessary:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--config', type=str, required=True)
args = parser.parse_args()
ADD REPLY
0
Entering edit mode

When i have the following code in the script :

if config["assembly"] == "megahit":
     shell ("megahit .........")
elif config["assembly"] == "metaspades":
     shell ("metaspades.py ........")

I am getting the following error:

if config["assembly"] == "megahit": NameError: name "config" is not defined

and I execute my pipeline like this

snakemake --cores 5 --config assembly="megahit"
ADD REPLY
0
Entering edit mode

config won't exist unless you have a configfile. do you have one?

ADD REPLY
0
Entering edit mode

May be gb 's answer to your earlier post is useful to you: Snakemake command line arguments.

ADD REPLY
0
Entering edit mode

By using the run directive, you cannot use conda environments... I need to load my programs from conda

ADD REPLY
0
Entering edit mode

You could do something like (unstested and quickly typed so only for inspiration):

rule assembly:
input:
    "trim/{sample}_1.fastq.gz",
    "trim/{sample}_2.fastq.gz",
output:
    "assembly/{sample}.fa"
params:
    method = config["assembly"]
conda: "environment.yaml"
script:
    "assembly.py"

And in the script:


if snakemake.params.method == "megahit":
    shell ("megahit .........")
elif snakemake.params.method == "metaspades":
    shell ("metaspades.py ........")

But I personally prefer something you are already doing:

rule assembly:
input:
    "trim/{sample}_1.fastq.gz",
    "trim/{sample}_2.fastq.gz",
output:
    "assembly/{sample}.fa"
params:
    fastq1 = "trim/{sample}_1.fastq.gz",
    fastq2 = "trim/{sample}_2.fastq.gz",
conda: "environment.yaml"
script:
    'assembly.py --assembly config["assembly"]'

and

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--assembly', dest='assembly' type=str, required=True)
args = parser.parse_args()

if args.assembly== "megahit":
    #guess you want subprocess.Popen
    shell ("megahit .........")
elif args.assembly== "metaspades":
    shell ("metaspades.py ........")

The above is just to push you in the right direction. If you can't figure it out I can make a working example. Also the shell line in the python script looks weird but for now I assume it is just a example or not real code.

ADD REPLY
0
Entering edit mode

Hello!

Thank you for the reply. Yes, the shell directive is an example and not the real code.

I will try your suggestion and I will get back to you.

ADD REPLY
3
Entering edit mode
2.9 years ago
Eric Lim ★ 2.2k

argparse is not needed.

(base) [~/Downloads/scratch/biostar/tmp]$ cat Snakefile 
rule:
    params:
        test = config['test']
    script:
        'test.py'

You can access config or params in your script with the snakemake object.

(base) [~/Downloads/scratch/biostar/tmp]$ cat test.py 
print(snakemake.config)

print(snakemake.params)

This works for config or configfile.

(base) [~/Downloads/scratch/biostar/tmp]$ snakemake -j1 --config test=hello
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
1            1              1              1
total        1              1              1

Select jobs to execute...

[Tue Jan 18 13:24:23 2022]
rule 1:
    jobid: 0
    resources: tmpdir=/var/folders/q4/cwjcyc851ldgbw2v28txzxbh0000gn/T

{'test': 'hello'}
hello
[Tue Jan 18 13:24:25 2022]
Finished job 0.
1 of 1 steps (100%) done
ADD COMMENT
0
Entering edit mode

I wouldn't agree that you never need to pass arguments to a python script in a snakemake workflow.

What if the python script you're trying to run is actually a published program and you just want to run it with certain parameters? Would I then have to run it from the shell directive?

ADD REPLY

Login before adding your answer.

Traffic: 1504 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6