Question

Using snakemake to download split-reads by SRA number

1

Entering edit mode

2.2 years ago

Ivan ▴ 60

This is inspired by a comment on this post by /u/dariober:

Basically, you are asking snakemake to produce one log file per SRR id and this log file is produced by rule download_files. As a side product, download_files will give you the actual fastq files (things could be done differently in this respect but hopefully this will help...)

I have a list of SRA files that I need to download to split reads. The goal is to get from [SRR1, .....] a list of files SRR1_1.fastq, SRR1_2.fastq in a predetermined folder. To that extent, I wrote the following file:

SRA_MAPPING = read_dictionary()
SRAFILES = list(SRA_MAPPING.keys())[1:]

RawSampleFolderName="raw_samples" 
RawSampleFolder=RawSampleFolderName+"/"

rule all:
    input:
        expand("{RawSampleFolder}{srafiles}_1.fastq",srafiles=SRAFILES, RawSampleFolder=RawSampleFolder),
        expand("{RawSampleFolder}{srafiles}_2.fastq",srafiles=SRAFILES, RawSampleFolder=RawSampleFolder)

rule download_srafiles: 
    output:
        expand("{RawSampleFolder}{srafiles}_1.fastq",srafiles=SRAFILES, RawSampleFolder=RawSampleFolder),
        expand("{RawSampleFolder}{srafiles}_2.fastq",srafiles=SRAFILES, RawSampleFolder=RawSampleFolder)
    params:
        download_folder = RawSampleFolderName
    shell:
        "fasterq-dump {wildcards.srafiles} -O {params.download_folder}"

(This is a proof of concept, I'll dump the global variables in config as soon as I can). The nutshell is that I have a list of SRA files read from a list, and I have a preset download folder. I use fasterq-dump to get split read files - from SR1 to SR_1.fastq and SR_2.fastq, In a snakemake fashion, I'd like for the rule download_srafiles to have output to be fastq files. My previous solution was given in the linked posted, but said solution made a .log file as an output and retrieved files as a side effect - I'd like to skip the part where I need log file. Since all I have is a python list, I skip the input. Running the above file does not download samples. Instead I get the error:

'Wildcards' object has no attribute 'srafiles'

So what is it that I'm doing wrong?

snakemake sratoolkit • 1.5k views

ADD COMMENT • link 2.2 years ago by Ivan ▴ 60

score 3 · Accepted Answer · 2022-02-08

The reason you are getting this error message is that you have no wildcards in your download_srafiles rule. When you use expand(), the function simply replaces the values enclosed in {}'s, so no wildcards remain. This is a bit confusing, but keep in mind that {}'s within expand means "variable to replace", whereas in normal inputs/outputs {}'s indicate wildcards.
In fact, you shouldn't use expand in your second rule - this is already taken care of by the rule all. So the code for the second rule should look something like:

rule download_srafiles: 
    output:
        "%s{srafiles}_1.fastq" % RawSampleFolder,
        "%s{srafiles}_2.fastq" % RawSampleFolder
    params:
        download_folder = RawSampleFolderName
    shell:
        "fasterq-dump {wildcards.srafiles} -O {params.download_folder}"

Finally, I am not familiar with fasterq-dump, but I highly recommend Kingfisher, which can also download from ENA which is much faster.