Make parts of process run once and other parts as loop

Question

[rohitsatyam102]

Hi Everyone

I was trying to run nextclade on my data that involves getting the data-set (one time thing nextclade dataset) and then use that dataset as an input to run nextclade run on multiple files (loop over) to assign lineage. I wrote a nextflow script however I get the following error:

java.nio.file.NoSuchFileException: /home/subudhak/Documents/COVID_Project/shortread/dataset1_sabir_Aug/results/11_nextclade/output/5314062_S9_L001_gene_M.translation.fasta
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
        at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
        at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
        at java.nio.file.Files.readAttributes(Files.java:1737)
        at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
        at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
        at java.nio.file.FileTreeWalker.next(FileTreeWalker.java:372)
        at java.nio.file.Files.walkFileTree(Files.java:2706)
        at java.nio.file.Files.walkFileTree(Files.java:2742)
        at nextflow.file.FileHelper.deleteDir0(FileHelper.groovy:963)
        at nextflow.file.FileHelper.deletePath(FileHelper.groovy:955)
        at nextflow.processor.PublishDir.processFile(PublishDir.groovy:319)
        at nextflow.processor.PublishDir.safeProcessFile(PublishDir.groovy:292)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1268)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
        at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:1029)
        at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1012)
        at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:101)
        at nextflow.processor.PublishDir$_apply1_closure1.doCall(PublishDir.groovy:265)
        at nextflow.processor.PublishDir$_apply1_closure1.call(PublishDir.groovy)
        at groovy.lang.Closure.run(Closure.java:493)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

To circumvent this error I tried using maxForks argument to limit the number of job runs simultaneously, but I think it executes the nextclade dataset command every time it picks-up a fasta file. Below is the code and I wish to know if there is a better way to do this:

nextflow.enable.dsl=2
params.raw = "results/09_assembly/*fa"
params.outdir = "results/11_nextclade"
params.jobs = 1

process NEXTCLADE{
    publishDir "$params.outdir", mode: 'copy'
    maxForks params.jobs
    input:
    tuple val(sid), path(assembly)
    output:
    path "*"
    """
    nextclade dataset get --name 'sars-cov-2' --output-zip sars-cov-2.zip
    nextclade run --jobs ${params.jobs} --input-dataset sars-cov-2.zip --output-all=output/ ${assembly} -n ${sid}
    """
}

input= Channel.fromPath(params.raw).map { file -> tuple(file.simpleName, file) }


workflow{
NEXTCLADE(input)
}