Nextflow runs Successfully but output files are empty. How to debug?
2
0
Entering edit mode
2.2 years ago

Hi Everyone!!

I am trying to alter the reference fasta files with the vcfs that I have. For that I wrote the following nextflow script. It runs successfully but the output fasta files are empty

nextflow.enable.dsl=2
params.raw = "results/06_gatkresults/vcfs/*.vcf.gz"
params.ref = "$baseDir/results/00_indexes/bwaidx/sequence.fasta"
params.outdir="results/07_assembly"
params.bed="$baseDir/results/05_lowcoverageBed"


process COV{
    publishDir "$params.outdir"
    input:
    tuple val(sid), path(vcf)
    output:
    file "*.fa"
        file "*.fasta"
    script:
    """ 
    echo ${sid}
    bcftools consensus -f ${params.ref} ${vcf} | sed 's/NC_045512.2/${sid}/' > ${sid}.fa 
    bedtools maskfasta -fi ${sid}.fa -bed ${params.bed}/${sid}.bed -fo ${sid}.fasta
    """
}

reads_ch = Channel.fromPath(params.raw).map { file -> tuple(file.simpleName, file) }

workflow {
  COV(reads_ch)
}

All the files are of zero bytes. How to find out what's wrong?

N E X T F L O W  ~  version 21.10.6
Launching `07_assembly.nf` [curious_swartz] - revision: 684fccf78d
executor >  local (12)
[8f/141386] process > COV (4) [100%] 12 of 12 ✔
nextflow bcftools • 1.6k views
ADD COMMENT
0
Entering edit mode

and what's the content of

cat work/8f/141386*/.command.sh
cat work/8f/141386*/.command.out
cat work/8f/141386*/.command.err

also, retry with

set -x
set -o pipefail
echo ${sid}
bcftools version
bcftools consensus -f "${params.ref}" "${vcf}" | sed 's/NC_045512.2/${sid}/' > "${sid}.fa"
test -s "${sid}.fa"
bedtools maskfasta -fi ${sid}.fa -bed "${params.bed}/${sid}.bed" -fo "${sid}.fasta"
test -s "${sid}.fasta"
ADD REPLY
0
Entering edit mode

.command.sh can be found in the process folder in the work directory. You will need to navigate to the right folder. In the initial post this would be work/8f/141386.... It's a hidden file so you will need something like ls -la to see it.

ADD REPLY
0
Entering edit mode
2.2 years ago
  1. Shouldn't your output files actualy be a path if you want to do a glob like "*.fasta" ? Can't you define as ${sid}.fasta like in the script section ?
  2. Check your work directories and the logs (errors ?)
  3. from the work directories use the various hidden files prepended by a dot eg .command.sh or whatever. One of these can be run with the exact input you have linked in. You can test, modify and improve this command directly, applying it in this work directory, before you know what's going wrong and repair your full script
ADD COMMENT
0
Entering edit mode
  1. In the previous six scripts based on same template file worked just fine without any problem. But yes path I tried with output: path "*.fa" but the files are still empty.
  2. Here is the log file: https://gist.github.com/Rohit.
  3. I am not sure where to find .command.sh. Can you help?
ADD REPLY
0
Entering edit mode

Tried your suggestion 1 and still the files are empty

nextflow.enable.dsl=2
params.raw = "results/06_gatkresults/vcfs/*.vcf.gz"
params.ref = "$baseDir/results/00_indexes/bwaidx/sequence.fasta"
params.outdir="results/07_assembly"
params.bed="$baseDir/results/05_lowcoverageBed"


process COV{
        publishDir "$params.outdir"
        input:
        tuple val(sid), path(vcf)
        output:
        path "*.fa"
        script:
        file1= sid + '.fa'
        file2= sid + '.fasta'
        """
        echo ${sid}
        bcftools consensus -f ${params.ref} ${vcf} | sed 's/NC_045512.2/${sid}/' > $file1
        bedtools maskfasta -fi ${sid}.fa -bed ${params.bed}/${sid}.bed -fo $file2
        """
}

reads_ch = Channel.fromPath(params.raw).map { file -> tuple(file.simpleName, file) }

workflow {
  COV(reads_ch)
}
ADD REPLY
0
Entering edit mode
2.2 years ago

Okay!! So I tried just running the bcftools and I realized that it wasn't able to locate the index file .csi unlike other tools like bwa and gatk. So I have to use ${vcf.toRealPath()} instead of just ${vcf} and that solved the issue. Thanks anyway for helping me colindaven

ADD COMMENT

Login before adding your answer.

Traffic: 766 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6