Reduce/Select reads/lines ffrom files using terminal

Question

[dimitrischat]

Hello all. I have .fastq files but with many reads/lines , how can i reduse that number f.e. 10000 lines and get a new file? Only using the terminal, not with some program ( macOS ). i would like a way that i can use any type of file and be able to reduse to whatever number

[GenoMax]

I had answered this question yesterday in a different thread posted by OP: C: tophat 2 rna seq

Answer 1

[jonasmst]

You can use sed to remove lines from a file (which is what I think you want?). The following command removes lines 45000-55000 from yourfile.fastq and saves the result in a new file called shorterfile.fastq:

sed -e '45000,55000d' < yourfile.fastq > shorterfile.fastq

Or you can use head or tail to extract the first (head) or last (tail) number of lines from a file, e.g.:

head -n 10000 yourfile.fastq > newfile.fastq

Answer 2

[Brian Bushnell]

The BBMap package has a tool called Reformat which can do this:

reformat.sh in=file.fastq out=reduced.fastq samplereadstarget=10000

That will sample 10000 reads randomly from the file. It can also process paired fastq files at once (with the in1 and in2 flags), keeping pairs together.

Answer 3

[Pierre Lindenbaum]

split https://linux.die.net/man/1/split

see also : Split Fastq File In Small Fastq Files - Windows splitting fastq files output size issue

[dimitrischat]

the first link is about splitting, i want to select the first x lines from the fastq file

[GenoMax]

If you are going to sample reads then I suggest not selecting first "x" reads from a file. Those reads may represent bad parts of data since they would be starting at the edge of the flowcell. With old(er) Illumina data a lot of the initial reads in a file used to have N's/bad Q scores.

[Brian Bushnell]

In my experience, they still do... :)