Defuse Fusion Discovery With Gzipped Fastq Input

Question

[ndejay]

Hi,

I am running out of space on my server and wish to have deFuse software running with gzipped fastq input files. Previously, I decompressed them, expanding them sometimes 4-fold. There is no way for me to erase other things on the server. I have tried to follow the manual and setting the data_converter_1 in the configuration file but to no avail. Is there a more elegant way of doing this than decompressing and then at the end of the deFuse run, delete the decompressed files?

If I only had single reads, I could have done something to the effect of:

zcat input.fastq.gz | perl defuse.pl -c config.txt -

but I have pair-ended reads.

Anyone got suggestions? Thanks in advance.

Answer 1

[Sean Davis]

Untested, but in bash (and probably others), you can often use process substitution which avoids having to explicitly make FIFOs.

perl defuse.pl -c config.txt -1 <( zcat input1.fastq.gz ) -2 <( zcat input2.fastq.gz )

[ndejay]

Will both substituted processes run in parallel or serially?

[Sean Davis]

They will run in parallel and will "stream" so that the entire file is not read into memory.

Answer 2

[Devon Ryan]

You can always create named pipes:

mkfifo file1
mkfifo file2
zcat input1.fastq.gz > file1 &
zcat input2.fastq.gz > file2 &
perl defuse.pl -c config.txt -1 file1 -2 file2 ...

In general, something like that will work. Remember to delete the FIFOs when you're finished.