fastq-mcf Error: Error during file close, possible partial write, failing

Question

[Ting]

Hi,

I’m using fastq-mcf to remove adapter sequences from RNAseq data.

fastq-mcf -o ctrl_clean_1.fastq -o ctrl_clean_2.fastq -l 50 -q 15 -t 0.1 -C 1000000 adaptor.fasta ctrl_1.fastq ctrl_2.fastq

After 4-5 hours running, it came with this error message:

Error during file close, possible partial write, failing

And the resulting trimmed reads are much fewer than expected.

Can anybody please tell me what causes this error?

here is the full log message:

Scale used: 2.2
Phred: 33
Threshold used: 1001 out of 1000000
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
): counted 6658 at the 'end' of 'ctrl_1.fastq', clip set to 6
Files: 2
Error during file close, possible partial write, failing
Total reads: 63127418
Too short after clip: 148714
Clipped 'end' reads (ctrl_1.fastq): Count 854897, Mean: 17.01, Sd: 10.28
Trimmed 22687152 reads (ctrl_1.fastq) by an average of 2.56 bases on quality < 15
Trimmed 27687235 reads (ctrl_2.fastq) by an average of 2.61 bases on quality < 15

Many thanks,

Ting

[GenoMax]

Not answering your question but giving a suggestion. You may want to use a more current scan/trimming tool instead. bbduk from BBMap, cutadapt, trimmomatic would all be good choices.

[Ting]

Thank you for your advice, @genomax. I found that the problem is not due to the software itself. Thank you anyway.

Answer 1

[Ting]

I found that it's the low I/O performance of the azure virtual machine caused the problem, not the software itself. It's solved by moving the input files from blob storage to a mounted disk. I've also tried some current trimming tools. "fastp" seems a good choice.