Error: Help understand Trimmomatic ZLIB input stream error
0
0
Entering edit mode
3.1 years ago
James Reeve ▴ 130

I've been running Trimmomatic of paried fastq.gz files and come across a weird error. It's particularly odd since the program finished running and the output looks normal. Can someone please give me a simplified explanation of what this error means? Furthermore, should I just ignore the message and keep the output?

Command:

 java -jar \$TRIMMOMATIC PE -threads 12 -phred33 \
-trimlog file/path/trimlog.txt \
file/path/input_1.fastq.gz file/path/input_2.fastq.gz \
-baseout file/path/output.fastq.gz \
SLIDINGWINDOW:5:20 TRAILING:30 MINLEN:90


Error Message:

java.io.EOFException: Unexpected end of ZLIB input stream
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
Exception in thread "Thread-0" java.lang.RuntimeException: java.io.EOFException: Unexpected end of ZLIB input stream
Caused by: java.io.EOFException: Unexpected end of ZLIB input stream
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
... 1 more


Terminal Message:

Input Read Pairs: 62991000 Both Surviving: 23166679 (36.78%) Forward Only Surviving: 8892038 (14.12%) Reverse Only Surviving: 6210627 (9.86%) Dropped: 24721656 (39.25%)
TrimmomaticPE: Completed successfully


This data had sequencing problems, the low % surviving isn't surprising

Fastq Output (example):

@ERR407508.7847495 HWI-ST143:472:D0JDUACXX:4:1208:5586:161136/1
GTAACAGCCGTTTGACGTAACAAACTGAACTTGAGGTTAACTCCCGGATCATCCCGGGTCACGCGATGACAACGTAAGCTAATTTGATCCGTTGATACAGA
+

Trimmomatic • 2.4k views
0
Entering edit mode

is that a head or a tail of your fastq file?

I think the ZLIB error message is triggered by the fact that the streams are not opened or closed correctly, so there might be something missing at the end of your fastq file.

0
Entering edit mode

That is the head. I checked the tail, it also looks normal.

0
Entering edit mode

Have you checked the integrity of your fastq file? It may be corrupt.validateFiles from Jim Kent's utilities should work.

1
Entering edit mode

test your fastq files:

gunzip -t your.fastq.gz && echo "VALID"

0
Entering edit mode

Thanks Pierre, this is a useful command for the future. In this case it was VALID

0
Entering edit mode

Please use validateFiles to ensure you have a valid fastq file with no broken records. Error could be anywhere in the file. You can also try fastqvalidator for this purpose.

0
Entering edit mode

It took a while to download, but i managed to run fastqvalidator. No errors showed up.

I wasn't able to get validateFiles to work. The command keeps telling me cannot execute binary file.

0
Entering edit mode

Did you add execute permission to validateFiles by doing chmod a+x validateFiles? I had linked linux version. If you are using macOS then you should get this file. Did you try running trimmomatic again? Perhaps the first time was just a stray error, if the file checks out otherwise.

1
Entering edit mode

Yes I found an error with validateFiles. I accidentally used the output file, that's why no error showed up before before. Thanks for the help.

 gzip: (stdin): unexpected end of file
Error [file=Us_L1-1.fastq.gz, line=251964208]: quality not as long as sequence (101 bases) [BCCFFDD>>DFHFIIJJJJJJJJJJIJJJJGIJIJJJIJJIJJJIJJJJJIIIJJJJDEHJJIJHHHHHBDFDDEB@DDDDDDDBDDDDDDCDCC@:>CD]
Aborting .. found 1 error

0
Entering edit mode

If you are willing, I suggest that you give bbduk.sh from BBMap suite a try and see if that is able to complete the trimming. There is a guide here.

0
Entering edit mode

Maybe the fastq.gz files are corrupted? Did you run them through FastQC? Did you try zcat | head and zcat | tail on those files?

0
Entering edit mode

Just tried it. Running FastQC generates normal results and zcat produces the same output as the gunzip file.

0
Entering edit mode

How exactly do you run the original command? Do you run it in your terminal or send it to a compute cluster?

0
Entering edit mode

I ran it through terminal, in a while loop that ran through 30 different files. All the other files ran without error.