I am able to run a nextflow script from the terminal but it fails with the below error when I run it as a cron job.
Terminal command
singularity run variant.call.sif nextflow run ./pf.resistance.nf --sampleid ERR8704637 --outDir ./output/ --reads "./input/ERR8704637_{1,2}.fastq.gz" -resume
Cron job command
/usr/bin/singularity run $HOME/pfal_variant_calling/variant.sif nextflow run $HOME/pfal_variant_calling/pf.resistance.nf --sampleid ERR8704637 --outDir $HOME/pfal_variant_calling/output/ --reads "$HOME/pfal_variant_calling/input/ERR8704637_\{1,2\}.fastq.gz" -resume
Error log
mkdir: cannot create directory ‘/home/mambauser/.nextflow’: Read-only file system
Warning: Couldn't create cached classpath folder: /home/mambauser/.nextflow/tmp/launcher/next
flow-one_22.10.1/pc_name -- Maybe NXF_HOME is not writable?
Exception in thread "main" java.lang.ExceptionInInitializerError
at nextflow.cli.Launcher.init(Launcher.groovy:123)
at nextflow.cli.Launcher.<init>(Launcher.groovy:84)
at nextflow.cli.Launcher.main(Launcher.groovy:646)
Caused by: java.lang.IllegalStateException: Cannot create path '/home/mambauser/.nextflow' --
check file system access permission
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native M
ethod)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeCon
structorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Deleg
atingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499
)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:72)
at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstru
ctor.java:59)
at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callC
onstructor(ConstructorSite.java:84)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSite
Array.java:59)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCall
Site.java:263)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCall
Site.java:277)
at nextflow.Const.getHomeDir(Const.groovy:111)
at nextflow.Const.<clinit>(Const.groovy:51)
... 3 more
Yes, the error log makes sense because that directory is owned by root.
Nextflow creates the .nextflow directory in the directory in which it is launched which is my home directory which is mounted in the singularity container so why does it try to create it in the /home/mambauser directory of the container?
I have verified that the working directory when the container launches is my system's home directory.
Also, why does it not throw this error when launched from the command line? Perhaps there's some property of cron that I do not know.
Yes, the error log makes sense because that directory is owned by root.
Nextflow creates the .nextflow directory in the directory in which it is launched which is my home directory which is mounted in the singularity container so why does it try to create it in the /home/mambauser directory of the container?
I have verified that the working directory when the container launches is my system's home directory.
Also, why does it not throw this error when launched from the command line? Perhaps there's some property of cron that I do not know.
I know about the Nextflow -with-singularity option but I'm running on multiple files on a server and I"ll much rather have cron handle it so I don't have keep the connection to the server running the whole time.
As said, run Singularity via Nextflow as it is recommended on the documentation instead of launching Singularity manually. This should take care of it. The idea is that Nextflow itself runs outside of the container, and it only uses the container to launch the jobs (=the tools you use).
I know about the Nextflow -with-singularity option but I'm running on multiple files on a server and I"ll much rather have cron handle it so I don't have to keep the connection to the server running the whole time. Also, I can't update the Java on the server so my best option is to run Nextflow from the container.
You do not need a running connection. NExtflow can be put to background with
-bg
or by submitting the nextflow job to a scheduler, or by usingscreen
. Your choice what you do, but your non-standard way of doing this is likely cause of the issue.Thank you screen was what I needed.