Error demultiplexing with Picard

Question

[Trivas]

I'm trying to demultiplex an iSeq run using a script that I modified that I usually use for a large NextSeq run. For some reason, the ExtractIlluminaBarcodes function stops working during tile 14 and fails for tiles 15 and 16. I have no explanation for why this is happening. Here's the command (formatted so you can read it):

ExtractIlluminaBarcodes -BASECALLS_DIR /home/ubuntu/path_to_basecalls/BaseCalls/ -Lane 1 -READ_STRUCTURE 151T8B8B151T 
-BARCODE_FILE /home/ubuntu/path_to_barecode_file/barcodes.txt 
-METRICS_FILE /home/ubuntu/path_to_metrics/metrics_output.1.txt

I've tried retransferring the files from the instrument to my instance using aws s3 sync multiple times thinking maybe something got corrupted during the transfer to no avail.

Here's the progress

INFO    2022-04-19 21:01:57     ExtractIlluminaBarcodes Processing with 1 PerTileBarcodeExtractor(s).
INFO    2022-04-19 21:01:57     ExtractIlluminaBarcodes Extracting barcodes for tile 1101
INFO    2022-04-19 21:02:00     ExtractIlluminaBarcodes Extracting barcodes for tile 1102
INFO    2022-04-19 21:02:03     ExtractIlluminaBarcodes Extracting barcodes for tile 1103
INFO    2022-04-19 21:02:05     ExtractIlluminaBarcodes Extracting barcodes for tile 1104
INFO    2022-04-19 21:02:07     ExtractIlluminaBarcodes Extracting barcodes for tile 1105
INFO    2022-04-19 21:02:09     ExtractIlluminaBarcodes Extracting barcodes for tile 1106
INFO    2022-04-19 21:02:12     ExtractIlluminaBarcodes Extracting barcodes for tile 1107
INFO    2022-04-19 21:02:14     ExtractIlluminaBarcodes Extracting barcodes for tile 1108
INFO    2022-04-19 21:02:17     ExtractIlluminaBarcodes Extracting barcodes for tile 1109
INFO    2022-04-19 21:02:19     ExtractIlluminaBarcodes Extracting barcodes for tile 1110
INFO    2022-04-19 21:02:21     ExtractIlluminaBarcodes Extracting barcodes for tile 1111
INFO    2022-04-19 21:02:24     ExtractIlluminaBarcodes Extracting barcodes for tile 1112
INFO    2022-04-19 21:02:26     ExtractIlluminaBarcodes Extracting barcodes for tile 1113
INFO    2022-04-19 21:02:28     ExtractIlluminaBarcodes Extracting barcodes for tile 1114
ERROR   2022-04-19 21:02:29     ExtractIlluminaBarcodes Error processing tile 1114
htsjdk.samtools.SAMFormatException: Did not inflate expected amount
        at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:147)
        at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:96)
        at htsjdk.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:550)
        at htsjdk.samtools.util.BlockCompressedInputStream.processNextBlock(BlockCompressedInputStream.java:532)
        at htsjdk.samtools.util.BlockCompressedInputStream.nextBlock(BlockCompressedInputStream.java:468)
        at htsjdk.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:458)
        at htsjdk.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:196)
        at htsjdk.samtools.util.BlockCompressedInputStream.read(BlockCompressedInputStream.java:331)
        at picard.illumina.parser.readers.BclReader.advance(BclReader.java:248)
        at picard.illumina.parser.readers.BclReader.hasNext(BclReader.java:202)
        at picard.illumina.parser.MultiTileBclParser$CountLimitedIterator.hasNext(MultiTileBclParser.java:111)
        at picard.illumina.parser.MultiTileBclParser$MultiTileBclDataCycleFileParser.hasNext(MultiTileBclParser.java:153)
        at picard.illumina.parser.PerTileCycleParser.hasNext(PerTileCycleParser.java:162)
        at picard.illumina.parser.MultiTileBclParser.hasNext(MultiTileBclParser.java:38)
        at picard.illumina.parser.IlluminaDataProvider.hasNext(IlluminaDataProvider.java:94)
        at picard.illumina.ExtractIlluminaBarcodes$PerTileBarcodeExtractor.run(ExtractIlluminaBarcodes.java:363)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
INFO    2022-04-19 21:02:29     ExtractIlluminaBarcodes Extracting barcodes for tile 1115
ERROR   2022-04-19 21:02:29     ExtractIlluminaBarcodes Error processing tile 1115
htsjdk.samtools.SAMFormatException: Invalid GZIP header
        at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:121)
        at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:96)
        at htsjdk.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:550)
        at htsjdk.samtools.util.BlockCompressedInputStream.processNextBlock(BlockCompressedInputStream.java:532)
        at htsjdk.samtools.util.BlockCompressedInputStream.nextBlock(BlockCompressedInputStream.java:468)
        at htsjdk.samtools.util.BlockCompressedInputStream.seek(BlockCompressedInputStream.java:380)
        at picard.illumina.parser.readers.BclReader.seek(BclReader.java:295)
        at picard.illumina.parser.MultiTileBclParser.makeReader(MultiTileBclParser.java:60)
        at picard.illumina.parser.MultiTileBclParser.access$000(MultiTileBclParser.java:38)
        at picard.illumina.parser.MultiTileBclParser$MultiTileBclDataCycleFileParser.<init>(MultiTileBclParser.java:134)
        at picard.illumina.parser.MultiTileBclParser.makeCycleFileParser(MultiTileBclParser.java:77)
        at picard.illumina.parser.MultiTileBclParser.makeCycleFileParser(MultiTileBclParser.java:71)
        at picard.illumina.parser.PerTileCycleParser.seekToTile(PerTileCycleParser.java:133)
        at picard.illumina.parser.MultiTileBclParser.seekToTile(MultiTileBclParser.java:38)
        at picard.illumina.parser.MultiTileBclParser.initialize(MultiTileBclParser.java:53)
        at picard.illumina.parser.MultiTileBclParser.<init>(MultiTileBclParser.java:47)
        at picard.illumina.parser.IlluminaDataProviderFactory.makeParser(IlluminaDataProviderFactory.java:394)
        at picard.illumina.parser.IlluminaDataProviderFactory.makeDataProvider(IlluminaDataProviderFactory.java:249)
        at picard.illumina.parser.IlluminaDataProviderFactory.makeDataProvider(IlluminaDataProviderFactory.java:228)
        at picard.illumina.ExtractIlluminaBarcodes$PerTileBarcodeExtractor.run(ExtractIlluminaBarcodes.java:355)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
INFO    2022-04-19 21:02:29     ExtractIlluminaBarcodes Extracting barcodes for tile 1116
ERROR   2022-04-19 21:02:30     ExtractIlluminaBarcodes Error processing tile 1116
htsjdk.samtools.SAMFormatException: Invalid GZIP header
        at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:121)
        at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:96)
        at htsjdk.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:550)
        at htsjdk.samtools.util.BlockCompressedInputStream.processNextBlock(BlockCompressedInputStream.java:532)
        at htsjdk.samtools.util.BlockCompressedInputStream.nextBlock(BlockCompressedInputStream.java:468)
        at htsjdk.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:458)
        at htsjdk.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:196)
        at htsjdk.samtools.util.BlockCompressedInputStream.seek(BlockCompressedInputStream.java:382)
        at picard.illumina.parser.readers.BclReader.seek(BclReader.java:295)
        at picard.illumina.parser.MultiTileBclParser.makeReader(MultiTileBclParser.java:60)
        at picard.illumina.parser.MultiTileBclParser.access$000(MultiTileBclParser.java:38)
        at picard.illumina.parser.MultiTileBclParser$MultiTileBclDataCycleFileParser.<init>(MultiTileBclParser.java:134)
        at picard.illumina.parser.MultiTileBclParser.makeCycleFileParser(MultiTileBclParser.java:77)
        at picard.illumina.parser.MultiTileBclParser.makeCycleFileParser(MultiTileBclParser.java:71)
        at picard.illumina.parser.PerTileCycleParser.seekToTile(PerTileCycleParser.java:133)
        at picard.illumina.parser.MultiTileBclParser.seekToTile(MultiTileBclParser.java:38)
        at picard.illumina.parser.MultiTileBclParser.initialize(MultiTileBclParser.java:53)
        at picard.illumina.parser.MultiTileBclParser.<init>(MultiTileBclParser.java:47)
        at picard.illumina.parser.IlluminaDataProviderFactory.makeParser(IlluminaDataProviderFactory.java:394)
        at picard.illumina.parser.IlluminaDataProviderFactory.makeDataProvider(IlluminaDataProviderFactory.java:249)
        at picard.illumina.parser.IlluminaDataProviderFactory.makeDataProvider(IlluminaDataProviderFactory.java:228)
        at picard.illumina.ExtractIlluminaBarcodes$PerTileBarcodeExtractor.run(ExtractIlluminaBarcodes.java:355)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
INFO    2022-04-19 21:02:30     ExtractIlluminaBarcodes Processed 16 tiles.
ERROR   2022-04-19 21:02:30     ExtractIlluminaBarcodes Abandoning metrics calculation because one or more PerTileBarcodeExtractors failed.

Does anyone have a suggestion of what to try? This command works perfectly for larger runs but for some reason is not scaling down like I'd expect it to.

Answer 1

[GenoMax]

Your file may be corrupt. Do you have a different copy you can try?

[Trivas]

I retransferred the data again and it seemed to fix everything. Weird, but glad it was an easy solution.