bcl2fastq ERROR: Could not parse the CSV stream text

Question

[tania.valdiviac]

Hi, I´m running bcl2fastq v2.20.0.422 and I am getting this error:

     ERROR: bcl2fastq::common::Exception: 2018-Aug-29 1$
            Dynamic exception type: boost::exception_detail::clone_impl<bcl2fastq::common::I$std::exception::what: Could not parse the CSV stream text:
        [HEADER],,,,,,,,
        Date,18/06/06,,,,,,,
        Workflow,GenerateFASTQ,,,,,,,
        Investigator,TaniaValdivia,,,,,,,
        Experiment,edna,,,,,,,
        ,,,,,,,,
        [Settings],,,,,,,,
        Adapter,CTGTCTCTTATACACATCT,,,,,,,
        ,,,,,,,,
        [Reads],,,,,,,,
        151,,,,,,,,
        151,,,,,,,,
        ,,,,,,,,
        [Data],,,,,,,,
 Lane,Sample_ID,Sample_Name,Sample_Project,i7_index_ID,index,i5_index_ID,index2,Amplicon
        ,1,1,Fish,iNext7_01_01,CTAGGTGA,iNext5_01_A,ACTGTGTC,12S Fish
        ,2,2,Fish,iNext7_01_02,ACGACTTG,iNext5_01_A,ACTGTGTC,12S Fish
        ,3,3,Fish,iNext7_01_03,GCATACAG,iNext5_01_A,ACTGTGTC,12S Fish
        ,4,4,Fish,iNext7_01_04,TGCGAACT,iNext5_01_A,ACTGTGTC,12S Fish
        ,5,5,Fish,iNext7_01_05,TCTCCGAT,iNext5_01_A,ACTGTGTC,12S Fish
        ,6,6,Fish,iNext7_01_06,GAGGACTT,iNext5_01_A,ACTGTGTC,12S Fish
        ,7,7,Fish,iNext7_01_07,ACCATCCA,iNext5_01_A,ACTGTGTC,12S Fish
        ,8,8,Fish,iNext7_01_08,CAACACCT,iNext5_01_A,ACTGTGTC,12S Fish
        ,9,9,Fish,iNext7_01_09,AGTTCGTC,iNext5_01_A,ACTGTGTC,12S Fish
        ,10,10,Fish,iNext7_01_10,CGAAGAAC,iNext5_01_A,ACTGTGTC,12S Fish
        ,11,11,Fish,iNext7_01_11,ATGGCGAA,iNext5_01_A,ACTGTGTC,12S Fish
        ,12,12,Fish,iNext7_01_12,TGGAGTTG,iNext5_01_A,ACTGTGTC,12S Fish
        ,13,13,Fish,iNext7_01_01,CTAGGTGA,iNext5_01_B,CGTTATGC,12S Fish
        ,14,14,Fish,iNext7_01_02,ACGACTTG,iNext5_01_B,CGTTATGC,12S Fish
        ,15,15,Fish,iNext7_01_03,GCATACAG,iNext5_01_B,CGTTATGC,12S Fish
        ,16,16,Fish,iNext7_01_04,TGCGAACT,iNext5_01_B,CGTTATGC,12S Fish
        ,17,17,Fish,iNext7_01_05,TCTCCGAT,iNext5_01_B,CGTTATGC,12S Fish
        ,18,18,Fish,iNext7_01_06,GAGGACTT,iNext5_01_B,CGTTATGC,12S Fish
        ,19,19,Fish,iNext7_01_07,ACCATCCA,iNext5_01_B,CGTTATGC,12S Fish
        ,20,20,Fish,iNext7_01_08,CAACACCT,iNext5_01_B,CGTTATGC,12S Fish
        ,21,21,Fish,iNext7_01_09,AGTTCGTC,iNext5_01_B,CGTTATGC,12S Fish
        ,22,22,Fish,iNext7_01_10,CGAAGAAC,iNext5_01_B,CGTTATGC,12S Fish
        ,23,23,Fish,iNext7_01_11,ATGGCGAA,iNext5_01_B,CGTTATGC,12S Fish
        ,24,24,Fish,iNext7_01_12,TGGAGTTG,iNext5_01_B,CGTTATGC,12S Fish
        ,25,25,Fish,iNext7_01_01,CTAGGTGA,iNext5_01_C,ACCTCTGT,12S Fish
        ,26,26,Fish,iNext7_01_02,ACGACTTG,iNext5_01_C,ACCTCTGT,12S Fish
        ,27,27,Fish,iNext7_01_03,GCATACAG,iNext5_01_C,ACCTCTGT,12S Fish
        ,28,28,Fish,iNext7_01_04,TGCGAACT,iNext5_01_C,ACCTCTGT,12S Fish
        ,29,29,Fish,iNext7_01_05,TCTCCGAT,iNext5_01_C,ACCTCTGT,12S Fish
        ,30,30,Fish,iNext7_01_06,GAGGACTT,iNext5_01_C,ACCTCTGT,12S Fish
    ,31,31,Fish,iNext7_01_07,ACCATCCA,iNext5_01_C,ACCTCTGT,12S Fish
    ,32,32,Eukaryotic,iNext7_01_08,CAACACCT,iNext5_01_C,ACCTCTGT,18S Eukaryotic
    ,33,33,Eukaryotic,iNext7_01_09,AGTTCGTC,iNext5_01_C,ACCTCTGT,18S Eukaryotic
    ,34,34,Eukaryotic,iNext7_01_10,CGAAGAAC,iNext5_01_C,ACCTCTGT,18S Eukaryotic
    ,35,35,Eukaryotic,iNext7_01_11,ATGGCGAA,iNext5_01_C,ACCTCTGT,18S Eukaryotic
    ,36,36,Eukaryotic,iNext7_01_12,TGGAGTTG,iNext5_01_C,ACCTCTGT,18S Eukaryotic
    ,37,37,Eukaryotic,iNext7_01_01,CTAGGTGA,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,38,38,Eukaryotic,iNext7_01_02,ACGACTTG,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,39,39,Eukaryotic,iNext7_01_03,GCATACAG,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,40,40,Eukaryotic,iNext7_01_04,TGCGAACT,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,41,41,Eukaryotic,iNext7_01_05,TCTCCGAT,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,42,42,Eukaryotic,iNext7_01_06,GAGGACTT,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,43,43,Eukaryotic,iNext7_01_07,ACCATCCA,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,44,44,Eukaryotic,iNext7_01_08,CAACACCT,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,45,45,Eukaryotic,iNext7_01_09,AGTTCGTC,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,46,46,Eukaryotic,iNext7_01_10,CGAAGAAC,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,47,47,Eukaryotic,iNext7_01_11,ATGGCGAA,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,48,48,Eukaryotic,iNext7_01_12,TGGAGTTG,iNext5_01_D,CTTAGTGG,18S Eukaryotic
    ,49,49,Eukaryotic,iNext7_01_01,CTAGGTGA,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,50,50,Eukaryotic,iNext7_01_02,ACGACTTG,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,51,51,Eukaryotic,iNext7_01_03,GCATACAG,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,52,52,Eukaryotic,iNext7_01_04,TGCGAACT,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,53,53,Eukaryotic,iNext7_01_05,TCTCCGAT,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,54,54,Eukaryotic,iNext7_01_06,GAGGACTT,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,55,55,Eukaryotic,iNext7_01_07,ACCATCCA,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,56,56,Eukaryotic,iNext7_01_08,CAACACCT,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,57,57,Eukaryotic,iNext7_01_09,AGTTCGTC,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,58,58,Eukaryotic,iNext7_01_10,CGAAGAAC,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,59,59,Eukaryotic,iNext7_01_11,ATGGCGAA,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,60,60,Eukaryotic,iNext7_01_12,TGGAGTTG,iNext5_01_E,ACGGAACA,18S Eukaryotic
    ,61,61,Fish,iNext7_01_01,CTAGGTGA,iNext5_01_F,CAGGTATC,12S Fish
    ,62,62,Fish,iNext7_01_02,ACGACTTG,iNext5_01_F,CAGGTATC,12S Fish
    ,63,63,Eukaryotic,iNext7_01_03,GCATACAG,iNext5_01_F,CAGGTATC,12S Fish
    ,64,64,Fish,iNext7_01_04,TGCGAACT,iNext5_01_F,CAGGTATC,12S Fish
    ,65,65,Fish,iNext7_01_05,TCTCCGAT,iNext5_01_F,CAGGTATC,12S Fish
    ,66,66,Fish,iNext7_01_06,GAGGACTT,iNext5_01_F,CAGGTATC,12S Fish
    ,67,67,Fish,iNext7_01_07,ACCATCCA,iNext5_01_F,CAGGTATC,12S Fish
    ,68,68,Fish,iNext7_01_08,CAACACCT,iNext5_01_F,CAGGTATC,12S Fish
    ,69,69,Fish,iNext7_01_09,AGTTCGTC,iNext5_01_F,CAGGTATC,12S Fish
    ,70,70,Fish,iNext7_01_10,CGAAGAAC,iNext5_01_F,CAGGTATC,12S Fish
    ,71,71,Fish,iNext7_01_11,ATGGCGAA,iNext5_01_F,CAGGTATC,12S Fish
    ,72,72,Fish,iNext7_01_12,TGGAGTTG,iNext5_01_F,CAGGTATC,12S Fish
    ,73,73,Fish,iNext7_01_01,CTAGGTGA,iNext5_01_G,TTACGGCT,12S Fish
    ,74,74,Fish,iNext7_01_02,ACGACTTG,iNext5_01_G,TTACGGCT,12S Fish
    ,75,75,Fish,iNext7_01_03,GCATACAG,iNext5_01_G,TTACGGCT,12S Fish
    ,76,76,Fish,iNext7_01_04,TGCGAACT,iNext5_01_G,TTACGGCT,12S Fish
    ,77,77,Fish,iNext7_01_05,TCTCCGAT,iNext5_01_G,TTACGGCT,12S Fish
    ,78,78,Fish,iNext7_01_06,GAGGACTT,iNext5_01_G,TTACGGCT,12S Fish
    ,79,79,Fish,iNext7_01_07,ACCATCCA,iNext5_01_G,TTACGGCT,12S Fish
    ,80,80,Fish,iNext7_01_08,CAACACCT,iNext5_01_G,TTACGGCT,12S Fish
    ,81,81,Fish,iNext7_01_09,AGTTCGTC,iNext5_01_G,TTACGGCT,12S Fish
    ,82,82,Fish,iNext7_01_10,CGAAGAAC,iNext5_01_G,TTACGGCT,12S Fish
    ,83,83,Fish,iNext7_01_11,ATGGCGAA,iNext5_01_G,TTACGGCT,12S Fish
    ,84,84,Fish,iNext7_01_12,TGGAGTTG,iNext5_01_G,TTACGGCT,12S Fish
    ,83,83,Fish,iNext7_01_11,ATGGCGAA,iNext5_01_G,TTACGGCT,12S Fish
    ,84,84,Fish,iNext7_01_12,TGGAGTTG,iNext5_01_G,TTACGGCT,12S Fish
    ,85,85,Fish,iNext7_01_01,CTAGGTGA,iNext5_01_H,TTCAGGAG,12S Fish
    ,86,86,Fish,iNext7_01_02,ACGACTTG,iNext5_01_H,TTCAGGAG,12S Fish
    ,87,87,Fish,iNext7_01_03,GCATACAG,iNext5_01_H,TTCAGGAG,12S Fish
    ,88,88,Eukaryotic,iNext7_01_04,TGCGAACT,iNext5_01_H,TTCAGGAG,12S Fish
    ,89,89,Fish,iNext7_01_05,TCTCCGAT,iNext5_01_H,TTCAGGAG,18S Eukaryotic
    ,90,90,Fish,iNext7_01_06,GAGGACTT,iNext5_01_H,TTCAGGAG,12S Fish
    ,91,91,Fish,iNext7_01_07,ACCATCCA,iNext5_01_H,TTCAGGAG,12S Fish
    ,92,92,Fish,iNext7_01_08,CAACACCT,iNext5_01_H,TTCAGGAG,12S Fish
    ,93,93,Fish,iNext7_01_09,AGTTCGTC,iNext5_01_H,TTCAGGAG,12S Fish
    ,94,94,Fish,iNext7_01_10,CGAAGAAC,iNext5_01_H,TTCAGGAG,12S Fish
    ,95,95,Fish,iNext7_01_11,ATGGCGAA,iNext5_01_H,TTCAGGAG,12S Fish
    ,96,96,Fish,iNext7_01_12,TGGAGTTG,iNext5_01_H,TTCAGGAG,12S Fish

The command that I am using is:

bcl2fastq -R ./180606_NB501110_0101_AH2FMVAFXY -o ./180606_NB501110_0101_AH2FMVAFXY/fastq2 --use-bases-mask y151,i8,i8,y151 --no-lane-splitting --create-fastq-for-index-reads

Could anyone help me?

Thanks a lot!

[GenoMax]

That SampleSheet is messed up. Why is there a blank field at the beginning of each row where there should be a lane number? I would start with fixing that first.

[tania.valdiviac]

Ok, sorry, I´m fixing it right now. It´s my first time running bcl2fastq and also my indexes are not comercial so I´m doing the SampleSheet it by hand.

[GenoMax]

You can use a tool (Windows only) that Illumina makes available to create samplesheets. It is easy to use and will ensure that the format is correct: https://support.illumina.com/sequencing/sequencing_software/experiment_manager/downloads.html

Since you are using custom indexes you can just make one dummy entry so you get the template for the right type of sequencer/kit etc. Then edit that file in Excel. Be sure to save as .csv text, once done.

[tania.valdiviac]

Thanks again Genomax and Devon Ryan.

As Genomax recommended, I managed to create my SampleSheet with the Illumina Software and then edited it, so now its working OK in bcl2fastq software :).

BUT, now I have another question. If I want to process my data from Lane 001 to 004, its correct if I write 1-4 in the Lane column?, or I need to do it Lane by Lane. This is because I´ve try this (writing 1-4), but it seems only to be working in L001 and L002 (the fastq files are created), but not for L003 and L004.

Greetings,

[Devon Ryan]

For a NextSeq you can eliminate the Lane column completely, you'll get all 4 lanes by default then.

Edit: I missed your other comment, looks like you already figured this out :)

[WouterDeCoster]

Please use ADD COMMENT or ADD REPLY to answer to previous reactions, as such this thread remains logically structured and easy to follow. I have now moved your reaction but as you can see it's not optimal. Adding an answer should only be used for providing a solution to the question asked.

[tania.valdiviac]

Thanks again Genomax and Devon Ryan. As Genomax recommended, I managed to create my SampleSheet with the Illumina Software and then edited it, so now its working OK in bcl2fastq software :). BUT, now I have another question. If I want to process my data from Lane 001 to 004, its correct if I write 1-4 in the Lane column?, or do I need to do it Lane by Lane. This is because I´ve try this (writing 1-4), but it seems only to be working in L001 and L002 (the fastq files are created), but not for L003 and L004. Greetings,

UPDATE: I found the answer in the manual, NOTE When the Lane column of the sample sheet Data section is populated, only those lanes are converted. When the Lane column is not used, all lanes are converted.

Answer 1

[Devon Ryan]

As Genomax indicated, remove the Lane column completely, both the (blank) contents and Lane, above them. You actually don't need anything except the [Data] section (in particular, keeping the adapter sequencing in the [Settings] section enable automatic adapter trimming, which you may not want).