Gnu Parallel Quoting With Awk
2
3
Entering edit mode
10.5 years ago
tayebwajb ▴ 120

As I try to get familiar with the GNU Parallel tutorial on quoting, I need some help to move faster.

The command I want to quote is

 parallel --dryrun "awk '{ print $0,"\t","{.}" }' {} > {}" ::: *foo.txt

Basically, I want to add the FILENAME as the last column of the tab-delimited file, for each input file.

Problem is that bash interpretes the above command as

awk '{ print /bin/bash,t,foo.txt }' foo.txt > foo.txt

Which doesn't work. So it needs appropriate quoting to work

parallel awk bash • 13k views
ADD COMMENT
10
Entering edit mode
10.1 years ago
ole.tange ★ 4.4k

I assume what you want to do is given the file foo.bar you want this run:

awk '{ print $0,"\t","foo" }' foo.bar > foo.bar

But that will not work, as you have foo.bar both as input an output. What will happen is that foo.bar will be 0 bytes long.

So you need a temporary file:

awk '{ print $0,"\t","foo" }' foo.bar > foo.bar.tmp; mv foo.bar.tmp foo.bar

To do that using parallel it might be easier to define a function:

doawk() {
   FILE=$1
   BASE=$2
   awk '{ print $0,"\t","'"$BASE"'" }' "$FILE" > "$FILE".tmp; mv "$FILE".tmp "$FILE"
}
export -f doawk
parallel doawk {} {.} ::: *.txt

This way you avoid double quoting the awk script.

If you want to avoid the function we are looking at:

parallel awk\ \'\{\ print\ \$0,\"\\t\",\"\'{.}\'\"\ \}\'\ {}\ \>\ {}.tmp\;\ mv\ {}.tmp\ {} ::: *.txt
ADD COMMENT
3
Entering edit mode
10.5 years ago
dapregi ▴ 50

With no info about the output expected, this is what I think you are trying to do:

parallel "awk '{print \$0, "\t", FILENAME}' {} > {}.tmp; mv {}.tmp {}" ::: *input.txt
ADD COMMENT
0
Entering edit mode

I added information about the expected output.

ADD REPLY

Login before adding your answer.

Traffic: 1554 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6