Entering edit mode
6.3 years ago
James Reeve
▴
130
I've been running Trimmomatic of paried fastq.gz files and come across a weird error. It's particularly odd since the program finished running and the output looks normal. Can someone please give me a simplified explanation of what this error means? Furthermore, should I just ignore the message and keep the output?
Command:
java -jar $TRIMMOMATIC PE -threads 12 -phred33 \
-trimlog file/path/trimlog.txt \
file/path/input_1.fastq.gz file/path/input_2.fastq.gz \
-baseout file/path/output.fastq.gz \
SLIDINGWINDOW:5:20 TRAILING:30 MINLEN:90
Error Message:
java.io.EOFException: Unexpected end of ZLIB input stream
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
at org.usadellab.trimmomatic.util.ConcatGZIPInputStream.read(ConcatGZIPInputStream.java:73)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:161)
at java.io.BufferedReader.readLine(BufferedReader.java:324)
at java.io.BufferedReader.readLine(BufferedReader.java:389)
at org.usadellab.trimmomatic.fastq.FastqParser.parseOne(FastqParser.java:85)
at org.usadellab.trimmomatic.fastq.FastqParser.next(FastqParser.java:179)
at org.usadellab.trimmomatic.threading.ParserWorker.run(ParserWorker.java:42)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "Thread-0" java.lang.RuntimeException: java.io.EOFException: Unexpected end of ZLIB input stream
at org.usadellab.trimmomatic.threading.ParserWorker.run(ParserWorker.java:56)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException: Unexpected end of ZLIB input stream
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
at org.usadellab.trimmomatic.util.ConcatGZIPInputStream.read(ConcatGZIPInputStream.java:73)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:161)
at java.io.BufferedReader.readLine(BufferedReader.java:324)
at java.io.BufferedReader.readLine(BufferedReader.java:389)
at org.usadellab.trimmomatic.fastq.FastqParser.parseOne(FastqParser.java:85)
at org.usadellab.trimmomatic.fastq.FastqParser.next(FastqParser.java:179)
at org.usadellab.trimmomatic.threading.ParserWorker.run(ParserWorker.java:42)
... 1 more
Terminal Message:
Input Read Pairs: 62991000 Both Surviving: 23166679 (36.78%) Forward Only Surviving: 8892038 (14.12%) Reverse Only Surviving: 6210627 (9.86%) Dropped: 24721656 (39.25%)
TrimmomaticPE: Completed successfully
This data had sequencing problems, the low % surviving isn't surprising
Fastq Output (example):
@ERR407508.7847495 HWI-ST143:472:D0JDUACXX:4:1208:5586:161136/1
GTAACAGCCGTTTGACGTAACAAACTGAACTTGAGGTTAACTCCCGGATCATCCCGGGTCACGCGATGACAACGTAAGCTAATTTGATCCGTTGATACAGA
+
@?@DF?DDHFHHFIHIGHIIJJEHJIIEHIGJJEGAD@FHHGJGHIHBEHIIGIGGEG9=ACCBBDBDCCDDDD?8??CACDDCDADC@C?<ABDDDDDCC
is that a
head
or atail
of your fastq file?I think the ZLIB error message is triggered by the fact that the streams are not opened or closed correctly, so there might be something missing at the end of your fastq file.
That is the head. I checked the tail, it also looks normal.
Have you checked the integrity of your fastq file? It may be corrupt.
validateFiles
from Jim Kent's utilities should work.test your fastq files:
Thanks Pierre, this is a useful command for the future. In this case it was
VALID
Please use
validateFiles
to ensure you have a valid fastq file with no broken records. Error could be anywhere in the file. You can also tryfastqvalidator
for this purpose.It took a while to download, but i managed to run
fastqvalidator
. No errors showed up.I wasn't able to get
validateFiles
to work. The command keeps telling mecannot execute binary file
.Did you add execute permission to
validateFiles
by doingchmod a+x validateFiles
? I had linked linux version. If you are using macOS then you should get this file. Did you try runningtrimmomatic
again? Perhaps the first time was just a stray error, if the file checks out otherwise.Yes I found an error with
validateFiles
. I accidentally used the output file, that's why no error showed up before before. Thanks for the help.If you are willing, I suggest that you give
bbduk.sh
from BBMap suite a try and see if that is able to complete the trimming. There is a guide here.Maybe the fastq.gz files are corrupted? Did you run them through FastQC? Did you try
zcat | head
andzcat | tail
on those files?Just tried it. Running FastQC generates normal results and
zcat
produces the same output as thegunzip
file.How exactly do you run the original command? Do you run it in your terminal or send it to a compute cluster?
I ran it through terminal, in a
while
loop that ran through 30 different files. All the other files ran without error.