Extract fasta files that are not empty from a directory and subdirectories

Question

[samlambrechts299]

Dear biostars,

I have a directory containing a few hundred subdirectories, each containing 7 fasta files. A significant fraction of these appear to be empty fasta files. Does anyone have an idea how to extract only those fasta sequences that are not empty?

Cheers,

Sam

[Joe]

Will they be truly empty, or will they have no sequence, but a header? Wouter's approach may not work if thats the case.

[WouterDeCoster]

You can use the unix find command to find empty files, see for example https://www.cyberciti.biz/faq/unix-linux-find-all-empty-files/

[cpad0112]

can you try this? samlambrechts299

 find . -type f -name "*.fa" -exec awk 'NR % 2 == 0 {if (length >=1) print FILENAME}' {} \;

Answer 1

[ATpoint]

A naive solution would be to use find to list all fasta files in the current and subdirectories and then simply check if beyond the headers they have any content:

function FindNonEmpty {

  if [[ $(grep -v '^>' $1 | head | awk NF | wc -l) > 0 ]]
    then
      $(realpath $1 >> not_empty.txt)
      else $(realpath $1 >> empty.txt)
    fi
}; export -f FindNonEmpty

find ./ -maxdepth 1000 -name "*.fa" | parallel FindNonEmpty {}

not_empty.txt will list all non-empty and empty.txt all empty files.