Question: Paralellize samtools piped with awk
gravatar for Korsocius
16 months ago by
Korsocius100 wrote:

Dear all,

I have question. I would like to parallelize one line command. Could you help me please?

ls 4*.bam | parallel --pipe -j 4 -k samtools view -h {} |  awk '(length($10) < 180 || $1 ~/^@/)' | awk '(length($10)>35 || $1 ~/^@/)' | samtools view -bS -o  '5{}'

Is it possible to do that like this or I have to separate samtools and awk? Thank you

samtools parallel • 477 views
ADD COMMENTlink modified 16 months ago • written 16 months ago by Korsocius100
gravatar for michael.ante
16 months ago by
michael.ante3.2k wrote:

You can write a shell script doing what your pipe is doing and using parallel with that script.

ADD COMMENTlink written 16 months ago by michael.ante3.2k

An expansion to Michael's script idea, you should do something like this:

ls 4*.bam | while read BAM
   [parallelised code executed on "${BAM}"]

Alternatively, you could split the output of ls into chunks and then run each in separate terminals:

ls 4*.bam | split --lines=4

This will create file 'chunks' called xaa xab xac, etc. You can run the same loop above (less parallel) multiple times in multiple sessions, and in this way it's parallelising in a different sense.

ADD REPLYlink written 16 months ago by Kevin Blighe39k
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 2552 users visited in the last hour