Filtering Whole Genome Sequence Data
2
0
Entering edit mode
2.7 years ago
Fid_o ▴ 40

I have whole genome sequence data (.FASTA files) for Salmonella bacteria. On average the sequence files have a size of 4.7MB but there are some that are too big, like 7Mb and others that are too small, like 500Kb. There is likelihood that the too large files contain unnecessary sequence data and the smaller ones have part of the organism genome sequenced, which would skew my data.

I would like to keep files that are in the range of 3.0- 6.0Mb. Am running on Linux server, any way around this?

Regards.

I would like to do gene core and accessory gene analysis on these bacterial genome sequences. I will be running roary after I do annotation. Or, based on what I want to do, what quality check can I do?

sequence genome • 492 views
1
Entering edit mode
2.7 years ago
mkdir -p keep && find dir1/dir2 -type f -name "*.fasta" -size +3M -size -6M -exec mv -v -t keep '{}' ';'

1
Entering edit mode

Ah, ok not surprising there are in-built function for size filtering. #TIL +1

0
Entering edit mode
2.7 years ago
ATpoint 66k

File size is a poor indicator for anything from my experience. Can you say what exactly you want to do and why, maybe there is a better alternatives to reach your goal. Please give some details.

If you type stat --printf="%s" your.file you get the size of a file in bytes. This you can then simply filter for your cutoffs. Say you have files that are all sufixed with *.fa we sort files within than cutoff to accept folder and outside of the range in reject folder:

mkdir accept
mkdir reject
ls *.fa \
do
if (( $(stat --printf="%s"$p) >= 3000000 )) && (( $(stat --printf="%s"$p) <= 6000000 ))
then mv $p accept/ else mv$p reject/
fi
done

0
Entering edit mode

Greetings @ATpoint!

I want to do core and accessory gene analysis using roary. I will later be doing phylogenetics. I want to remove all sequences that are too small or too large as, from what I was told, a too small size would mean only a shorter part of the genome was sequenced and too large a size would probably mean other things, like plasmids, might have been sequenced.

0
Entering edit mode

In this case I suggest you better count base pairs instead of file size. Something like this: Sequence length from Fasta I am not at all a genome assembly expert, so I only suggest a solution from the technical side, no idea if this makes sense in your context.