Nextflow - How to pass the yml format input file from an argument to channel as a list
3
0
Entering edit mode
14 months ago
kani ▴ 10

How to pass the yml format input file from an argument to a channel samples as a list? Appreciate if any nextflow function or simple solution to make the channel samples works with --input_list s3://my-bucket/params.yml instead of -params-file s3://my-bucket/params.yml

The given below channel samples code works with -params-file s3://my-bucket/params.yml but that's not I'm looking.

$ nextflow run main.nf \ -config nextflow.config \ --input_list s3://my-bucket/params.yml \ --publish_dir ./output

--input_list s3://my-bucket/params.yml

samples:
-
 biosample_id: WGS001
 bam: WGS001.bam
-
 biosample_id: WGS0002
 bam: NWGS0002.bam

workflow

Channel
    .fromList( params.samples )
    .ifEmpty { ['biosample_id': params.biosample_id, 'bam': params.bam] }
    .set { samples }


Channel
    samples.branch { rec ->
    ....

Channel
    samples.map { it.biosample_id }
    ....
python groovy yaml nextflow • 1.8k views
ADD COMMENT
1
Entering edit mode
14 months ago

I don't think there is a native NF operator to read YAML:

Nevertheless, nextflow itself contains a java library parsing yaml. org.yaml.snakeyaml.Yaml

So you can try:

ADD COMMENT
0
Entering edit mode
14 months ago
pbioinf ▴ 140

Use the undocumented -params-file option. Documentation is unfortunately lacking. The parsed yaml file becomes part of your parameters list in params.

From nextflow run -help:

-params-file
   Load script parameters from a JSON/YAML file
ADD COMMENT
1
Entering edit mode

How come undocumented? https://www.nextflow.io/docs/latest/cli.html?highlight=params%20file

There are even examples over there.

ADD REPLY
0
Entering edit mode

You are right, last time I looked into this it was still undocumented.

ADD REPLY
0
Entering edit mode

I want to pass the input file through an argument unfortunately not with params-file, reason being unable to use the params-file in the nftower dashboard under Pipeline parameters field which is equivalent to params-file option allowing to paste the json/yaml file in the field but it has a charcters limitation to give large number of samples list. To overcome this I need to give the input file with an arg like. insput_list: s3://my-bucket/params.yaml

ADD REPLY
1
Entering edit mode

I'd recommend to use a csv file to give out input file instead. This is what we do most of the time in nf-core nowadays.

ADD REPLY
0
Entering edit mode
14 months ago
kani ▴ 10

This works well...

import groovy.yaml.YamlSlurper

params.inputs_list = "inputs.yaml"

workflow {    
    inputs = new YamlSlurper().parse(file(params.inputs_list))

    Channel
        .fromList(inputs['samples'])
        .ifEmpty { ['biosample_id': params.biosample_id, 'aln': params.aln] }
        .set { samples }

https://github.com/nextflow-io/nextflow/discussions/4288

ADD COMMENT

Login before adding your answer.

Traffic: 1625 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6