Is it possible to pass command line arguments to python script in snakemake?
1
0
Entering edit mode
14 months ago

Hello all,

When I am executing my snakemake pipeline I am trying to pass command line arguments with the --config flag in a python script.

rule assembly:
input:
"trim/{sample}_1.fastq.gz",
"trim/{sample}_2.fastq.gz",
output:
"assembly/{sample}.fa"
params:
fastq1 = "trim/{sample}_1.fastq.gz",
fastq2 = "trim/{sample}_2.fastq.gz",
conda: "environment.yaml"
script:
"assembly.py"


The Python script

import argparse
parser = argparse.ArgumentParser()
args = parser.parse_args()

if args.config["assembly"] == "megahit":
shell ("megahit .........")


I am executing the pipeline like this:

snakemake --cores 5 --config assembly="megahit"


or

snakemake --cores 5 --config assembly="metaspades"


When I do that I am getting the following error:

error: the following arguments are required: --config

1. Is there a way to make this work?
2. Am I completely wrong by choosing this option? (I did because when I used the snakemake run directive I was getting the "conda environments are only allowed with shell script notebook or wrapper directives (not with run)" error.

snakemake python • 2.4k views
0
Entering edit mode

i don't think you need to add the addparse stuff, just use a generic configfile with a default value and your inline arguments should work

0
Entering edit mode

So, you suggest instead of --config use the --configfile and specify the different params in this one?

0
Entering edit mode

all this is unnecessary:

import argparse
parser = argparse.ArgumentParser()
args = parser.parse_args()

0
Entering edit mode

When i have the following code in the script :

if config["assembly"] == "megahit":
shell ("megahit .........")


I am getting the following error:

if config["assembly"] == "megahit": NameError: name "config" is not defined

and I execute my pipeline like this

snakemake --cores 5 --config assembly="megahit"

0
Entering edit mode

config won't exist unless you have a configfile. do you have one?

0
Entering edit mode

May be gb 's answer to your earlier post is useful to you: Snakemake command line arguments.

0
Entering edit mode

By using the run directive, you cannot use conda environments... I need to load my programs from conda

0
Entering edit mode

You could do something like (unstested and quickly typed so only for inspiration):

rule assembly:
input:
"trim/{sample}_1.fastq.gz",
"trim/{sample}_2.fastq.gz",
output:
"assembly/{sample}.fa"
params:
method = config["assembly"]
conda: "environment.yaml"
script:
"assembly.py"


And in the script:


if snakemake.params.method == "megahit":
shell ("megahit .........")


But I personally prefer something you are already doing:

rule assembly:
input:
"trim/{sample}_1.fastq.gz",
"trim/{sample}_2.fastq.gz",
output:
"assembly/{sample}.fa"
params:
fastq1 = "trim/{sample}_1.fastq.gz",
fastq2 = "trim/{sample}_2.fastq.gz",
conda: "environment.yaml"
script:
'assembly.py --assembly config["assembly"]'


and

import argparse
parser = argparse.ArgumentParser()
args = parser.parse_args()

if args.assembly== "megahit":
#guess you want subprocess.Popen
shell ("megahit .........")


The above is just to push you in the right direction. If you can't figure it out I can make a working example. Also the shell line in the python script looks weird but for now I assume it is just a example or not real code.

0
Entering edit mode

Hello!

Thank you for the reply. Yes, the shell directive is an example and not the real code.

I will try your suggestion and I will get back to you.

0
Entering edit mode
14 months ago
Eric Lim ★ 2.0k

argparse is not needed.

(base) [~/Downloads/scratch/biostar/tmp]$cat Snakefile rule: params: test = config['test'] script: 'test.py'  You can access config or params in your script with the snakemake object. (base) [~/Downloads/scratch/biostar/tmp]$ cat test.py
print(snakemake.config)

print(snakemake.params)


This works for config or configfile.

(base) [~/Downloads/scratch/biostar/tmp]\$ snakemake -j1 --config test=hello
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
-----  -------  -------------  -------------
1            1              1              1
total        1              1              1

Select jobs to execute...

[Tue Jan 18 13:24:23 2022]
rule 1:
jobid: 0
resources: tmpdir=/var/folders/q4/cwjcyc851ldgbw2v28txzxbh0000gn/T

{'test': 'hello'}
hello
[Tue Jan 18 13:24:25 2022]
Finished job 0.
1 of 1 steps (100%) done

0
Entering edit mode

I wouldn't agree that you never need to pass arguments to a python script in a snakemake workflow.

What if the python script you're trying to run is actually a published program and you just want to run it with certain parameters? Would I then have to run it from the shell directive?