Question: Gnu Parallel Quoting With Awk
1
gravatar for tayebwajb
4.3 years ago by
tayebwajb90
Sweden
tayebwajb90 wrote:

As I try to get familiar with the GNU Parallel tutorial on quoting, I need some help to move faster.

The command I want to quote is

 parallel --dryrun "awk '{ print $0,"\t","{.}" }' {} > {}" ::: *foo.txt

Basically, I want to add the FILENAME as the last column of the tab-delimited file, for each input file.

Problem is that bash interpretes the above command as

awk '{ print /bin/bash,t,foo.txt }' foo.txt > foo.txt

Which doesn't work. So it needs appropriate quoting to work

parallel awk bash • 5.3k views
ADD COMMENTlink modified 3.9 years ago by ole.tange3.1k • written 4.3 years ago by tayebwajb90
7
gravatar for ole.tange
3.9 years ago by
ole.tange3.1k
Denmark
ole.tange3.1k wrote:

I assume what you want to do is given the file `foo.bar` you want this run:

    awk '{ print $0,"\t","foo" }' foo.bar > foo.bar

But that will not work, as you have foo.bar both as input an output. What will happen is that foo.bar will be 0 bytes long.

So you need a temporary file:

    awk '{ print $0,"\t","foo" }' foo.bar > foo.bar.tmp; mv foo.bar.tmp foo.bar

To do that using parallel it might be easier to define a function:

    doawk() {
       FILE=$1
       BASE=$2
       awk '{ print $0,"\t","'"$BASE"'" }' "$FILE" > "$FILE".tmp; mv "$FILE".tmp "$FILE"
    }
    export -f doawk
    parallel doawk {} {.} ::: *.txt

This way you avoid double quoting the awk script.

If you want to avoid the function we are looking at:

    parallel awk\ \'\{\ print\ \$0,\"\\t\",\"\'{.}\'\"\ \}\'\ {}\ \>\ {}.tmp\;\ mv\ {}.tmp\ {} ::: *.txt

 

 

ADD COMMENTlink modified 3.9 years ago • written 3.9 years ago by ole.tange3.1k
3
gravatar for dapregi
4.3 years ago by
dapregi50
Spain
dapregi50 wrote:

With no info about the output expected, this is what I think you are trying to do:

parallel "awk '{print \$0, "\t", FILENAME}' {} > {}.tmp; mv {}.tmp {}" ::: *input.txt
ADD COMMENTlink written 4.3 years ago by dapregi50

I added information about the expected output.

ADD REPLYlink written 4.3 years ago by tayebwajb90
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 940 users visited in the last hour