Error: Help understand Trimmomatic ZLIB input stream error
0
0
Entering edit mode
3.1 years ago
James Reeve ▴ 130

I've been running Trimmomatic of paried fastq.gz files and come across a weird error. It's particularly odd since the program finished running and the output looks normal. Can someone please give me a simplified explanation of what this error means? Furthermore, should I just ignore the message and keep the output?

Command:

 java -jar $TRIMMOMATIC PE -threads 12 -phred33 \
   -trimlog file/path/trimlog.txt \
   file/path/input_1.fastq.gz file/path/input_2.fastq.gz \
   -baseout file/path/output.fastq.gz \
   SLIDINGWINDOW:5:20 TRAILING:30 MINLEN:90

Error Message:

java.io.EOFException: Unexpected end of ZLIB input stream
    at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
    at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
    at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
    at org.usadellab.trimmomatic.util.ConcatGZIPInputStream.read(ConcatGZIPInputStream.java:73)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
    at java.io.InputStreamReader.read(InputStreamReader.java:184)
    at java.io.BufferedReader.fill(BufferedReader.java:161)
    at java.io.BufferedReader.readLine(BufferedReader.java:324)
    at java.io.BufferedReader.readLine(BufferedReader.java:389)
    at org.usadellab.trimmomatic.fastq.FastqParser.parseOne(FastqParser.java:85)
    at org.usadellab.trimmomatic.fastq.FastqParser.next(FastqParser.java:179)
    at org.usadellab.trimmomatic.threading.ParserWorker.run(ParserWorker.java:42)
    at java.lang.Thread.run(Thread.java:748)
Exception in thread "Thread-0" java.lang.RuntimeException: java.io.EOFException: Unexpected end of ZLIB input stream
    at org.usadellab.trimmomatic.threading.ParserWorker.run(ParserWorker.java:56)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException: Unexpected end of ZLIB input stream
    at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
    at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
    at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
    at org.usadellab.trimmomatic.util.ConcatGZIPInputStream.read(ConcatGZIPInputStream.java:73)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
    at java.io.InputStreamReader.read(InputStreamReader.java:184)
    at java.io.BufferedReader.fill(BufferedReader.java:161)
    at java.io.BufferedReader.readLine(BufferedReader.java:324)
    at java.io.BufferedReader.readLine(BufferedReader.java:389)
    at org.usadellab.trimmomatic.fastq.FastqParser.parseOne(FastqParser.java:85)
    at org.usadellab.trimmomatic.fastq.FastqParser.next(FastqParser.java:179)
    at org.usadellab.trimmomatic.threading.ParserWorker.run(ParserWorker.java:42)
    ... 1 more

Terminal Message:

Input Read Pairs: 62991000 Both Surviving: 23166679 (36.78%) Forward Only Surviving: 8892038 (14.12%) Reverse Only Surviving: 6210627 (9.86%) Dropped: 24721656 (39.25%)
TrimmomaticPE: Completed successfully

This data had sequencing problems, the low % surviving isn't surprising

Fastq Output (example):

@ERR407508.7847495 HWI-ST143:472:D0JDUACXX:4:1208:5586:161136/1
GTAACAGCCGTTTGACGTAACAAACTGAACTTGAGGTTAACTCCCGGATCATCCCGGGTCACGCGATGACAACGTAAGCTAATTTGATCCGTTGATACAGA
+
@?@DF?DDHFHHFIHIGHIIJJEHJIIEHIGJJEGAD@FHHGJGHIHBEHIIGIGGEG9=ACCBBDBDCCDDDD?8??CACDDCDADC@C?<ABDDDDDCC
Trimmomatic • 2.4k views
ADD COMMENT
0
Entering edit mode

is that a head or a tail of your fastq file?

I think the ZLIB error message is triggered by the fact that the streams are not opened or closed correctly, so there might be something missing at the end of your fastq file.

ADD REPLY
0
Entering edit mode

That is the head. I checked the tail, it also looks normal.

ADD REPLY
0
Entering edit mode

Have you checked the integrity of your fastq file? It may be corrupt.validateFiles from Jim Kent's utilities should work.

ADD REPLY
1
Entering edit mode

test your fastq files:

gunzip -t your.fastq.gz && echo "VALID"
ADD REPLY
0
Entering edit mode

Thanks Pierre, this is a useful command for the future. In this case it was VALID

ADD REPLY
0
Entering edit mode

Please use validateFiles to ensure you have a valid fastq file with no broken records. Error could be anywhere in the file. You can also try fastqvalidator for this purpose.

ADD REPLY
0
Entering edit mode

It took a while to download, but i managed to run fastqvalidator. No errors showed up.

I wasn't able to get validateFiles to work. The command keeps telling me cannot execute binary file.

ADD REPLY
0
Entering edit mode

Did you add execute permission to validateFiles by doing chmod a+x validateFiles? I had linked linux version. If you are using macOS then you should get this file. Did you try running trimmomatic again? Perhaps the first time was just a stray error, if the file checks out otherwise.

ADD REPLY
1
Entering edit mode

Yes I found an error with validateFiles. I accidentally used the output file, that's why no error showed up before before. Thanks for the help.

 gzip: (stdin): unexpected end of file    
 Error [file=Us_L1-1.fastq.gz, line=251964208]: quality not as long as sequence (101 bases) [BCCFFDD>>DFHFIIJJJJJJJJJJIJJJJGIJIJJJIJJIJJJIJJJJJIIIJJJJDEHJJIJHHHHHBDFDDEB@DDDDDDDBDDDDDDCDCC@:>CD]
Aborting .. found 1 error
ADD REPLY
0
Entering edit mode

If you are willing, I suggest that you give bbduk.sh from BBMap suite a try and see if that is able to complete the trimming. There is a guide here.

ADD REPLY
0
Entering edit mode

Maybe the fastq.gz files are corrupted? Did you run them through FastQC? Did you try zcat | head and zcat | tail on those files?

ADD REPLY
0
Entering edit mode

Just tried it. Running FastQC generates normal results and zcat produces the same output as the gunzip file.

ADD REPLY
0
Entering edit mode

How exactly do you run the original command? Do you run it in your terminal or send it to a compute cluster?

ADD REPLY
0
Entering edit mode

I ran it through terminal, in a while loop that ran through 30 different files. All the other files ran without error.

ADD REPLY

Login before adding your answer.

Traffic: 1007 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6