snakemake
1
0
Entering edit mode
9 weeks ago

Hi

I'm having an issue in my snakemake file it is running but there is an issue where it shows missing output exception

SRA,FRR = glob_wildcards("rawReads/{sra}_{frr}.fastq.gz")

rule all:

input:
expand("rawQC/{sra}_{frr}_fastqc.{extension}", sra=SRA, frr=FRR,extension=["gz","html"]),

rule rawFastqc:

input:
output:
gz="rawQC/{sra}_{frr}_fastqc.gz",
html="rawQC/{sra}_{frr}_fastqc.html",
1
params:
path="rawQC/",
shell:
"""
"""

rule fastp:

input:
output:
4
shell:
"""
"""


and this is the error

MissingOutputException in line 10 of /mnt/d/snakemake/snakefile.py:                                                                                                     Job Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:                rawQC/SRR8571278_2_fastqc.gz completed successfully, but some output files are missing. 2                                                                               Removing output files of failed job rawFastqc since they might be corrupted:


and I'm trying to solve the error but I'm confused on what is missing in the code

fastp snakemake • 403 views
0
Entering edit mode

it sounds like the files are not found, fastqc runs with invalid input, errors out in some way, and evidently the output files are also missing

Personal observation: in general for examples like this snake make is overkill, I would suggest learning something simpler first, bash scripting, GNU parallel, simple Makefiles etc.

0
Entering edit mode

This worked for me. Code is copy/pasted from your code:

directory:

$tree rawReads rawReads ├── sra_frr.fastq.gz └── test.sm 0 directories, 2 files  snakemake file: $ cat rawReads/test.sm

print (SRA)
print(FRR)

rule all:
input:
expand("rawQC/{sra}_{frr}_fastqc.{extension}", sra=SRA, frr=FRR,extension=["gz","html"]),

rule rawFastQC:
input:
output:
gz="rawQC/{sra}_{frr}_fastqc.gz",
html="rawQC/{sra}_{frr}_fastqc.html"
1
params:
path="rawQC/"
shell:
"""
"""


code run:

\$ snakemake -ns rawReads/test.sm -j 1

['sra']
['frr']
Building DAG of jobs...
Job stats:
---------  -------  -------------  -------------
all              1              1              1
rawFastQC        1              1              1
total            2              1              1

[Mon May  2 12:02:42 2022]
rule rawFastQC:
output: rawQC/sra_frr_fastqc.gz, rawQC/sra_frr_fastqc.html
jobid: 1
wildcards: sra=sra, frr=frr
resources: tmpdir=/var/folders/w6/z_3lbbdx0j7_s1wx3bt31n6c0000gn/T

[Mon May  2 12:02:42 2022]
localrule all:
input: rawQC/sra_frr_fastqc.gz, rawQC/sra_frr_fastqc.html
jobid: 0
resources: tmpdir=/var/folders/w6/z_3lbbdx0j7_s1wx3bt31n6c0000gn/T

Job stats:
---------  -------  -------------  -------------
all              1              1              1
rawFastQC        1              1              1
total            2              1              1

This was a dry-run (flag -n). The order of jobs does not reflect the order of execution

0
Entering edit mode

Hii thanks for the suggestion but in dry run it is working but whenever I try to run it the whole analysis is completing but in last again the same error occurs

1
Entering edit mode
9 weeks ago

There is fundamental mistake. Fastqc doesn't output gz. It outputs zip and html. That is why it's failing.

Try this script:

SRA,FRR = glob_wildcards("rawREADS/{sra}_{frr}.fastq.gz")
print (SRA)
print(FRR)

rule all:
input:
expand("rawQC/{sra}_{frr}_fastqc.{extension}", sra=SRA, frr=FRR,extension=["zip","html"]),

rule rawFastQC:
input:
output:
zip="rawQC/{sra}_{frr}_fastqc.zip",
html="rawQC/{sra}_{frr}_fastqc.html"
1
params:
path="rawQC/"
shell:
"""
"""


beware of copy/pasting tabs.

0
Entering edit mode

Thanks for making me understand it works now I thought that as my input is .gz format then the output is also like that

Really big thank you you cleared my concept of how snakemake works