Question: Bash $@ (Dollar At Variable) Loses Quote Characters
1
gravatar for Peter
2.1 years ago by
Peter3.8k
Scotland, UK
Peter3.8k wrote:

Here's a challenge for the shell script experts, something which I have not found easy to find via Google: Is there a quote/escaping aware version of bash's $@ syntax?

I have a simple bash wrapper script where I use the $@ variable to get all the command line variables passed to the script. I'm actually using this to run BLAST on our cluster but first automatically cache databases on the local node's hard drive, and update the paths in the command line arguments to match. However, the following reduced script demonstrates the problem:

#!/bin/bash
echo Timing this: $@
time $@

Save that as time_it and make it executable, and it can be used as follows:

$ ./time_it /xxx/blastp -outfmt 6 -query x.faa -db nr -task blastp -out x.tsv
Timing this: /xxx/blastp -outfmt 6 -query x.faa -db nr -task blastp -out x.tsv
...

real    0m0.021s
user    0m0.012s
sys    0m0.005s

However, a command like this fails:

$ ./time_it /xxx/blastp -outfmt "6 std score" -query x.faa -db nr -task blastp -out x.tsv
Timing this: /xxx/blastp -outfmt 6 std score -query x.faa -db nr -task blastp -out x.tsv
USAGE
...
Too many positional arguments (1), the offending value: std
...

real    0m0.026s
user    0m0.012s
sys    0m0.005s

Here the third argument is "6 std score", quoted to keep it as one rather than three arguments, but using $@ in the bash script lost the quote marks (as shown via the echo time, and the error message from BLAST).

I can avoid this by switching from bash to something else like Python, but I'm hoping there is a neat bash (or sh) solution instead.

ADD COMMENTlink modified 22 months ago by Julian180 • written 2.1 years ago by Peter3.8k
5
gravatar for brentp
2.1 years ago by
brentp17k
Denver, Colorado
brentp17k wrote:

can you use use quote, e.g. "$@" ?

time.sh

time "$@"

t.py

import sys
print "|".join(sys.argv[1:])

then run as:

bash time.sh python t.py a b "c and d"

and get:

timing this python t.py a b c and d

a|b|c and d

real    0m0.043s
user    0m0.032s
sys 0m0.010s
ADD COMMENTlink written 2.1 years ago by brentp17k
1

See http://www.grymoire.com/Unix/Sh.html#uh-42 for an overview of the various parameter variables, including '$@' and '$*' used by Bourne shells.

ADD REPLYlink written 2.1 years ago by Hamish2.4k

Curious that this quote trick doesn't work on the echo line.

ADD REPLYlink written 2.1 years ago by Peter3.8k
3
gravatar for Istvan Albert
2.1 years ago by
Istvan Albert ♦♦ 39k
University Park, USA
Istvan Albert ♦♦ 39k wrote:

I think the problem is not with the content of $@ but rather the behavior of the last line that transforms it into a command, this below works for me:

#!/bin/bash
echo Timing this: $@
time exec "$@"

invoking it as:

$ ./time_it python test.py "ONE TWO THREE" "FOUR FIVE"
Timing this: python test.py ONE TWO THREE FOUR FIVE
ONE TWO THREE
FOUR FIVE

test program:

$ more test.py 
import sys
print sys.argv[1]
print sys.argv[2]
ADD COMMENTlink written 2.1 years ago by Istvan Albert ♦♦ 39k
1

+1 exactly my thought on illustrating. just faster :)

ADD REPLYlink written 2.1 years ago by brentp17k

Including exec made it a little more complicated than Brent's answer.

This form with exec at the start seems like another solution that should work with other commands (time is special as it evaluates the rest of the command itself):

exec time $@
ADD REPLYlink written 2.1 years ago by Peter3.8k

Including exec made it a little more complicated than Brent's answer. This form with exec at the start seems like another solution that should work with other commands (time is special as it evaluates the rest of the command itself): exec time "$@"

ADD REPLYlink written 2.1 years ago by Peter3.8k
2
gravatar for Julian
2.1 years ago by
Julian180
Manchester, UK
Julian180 wrote:

Could you not call it in the following manner? This way you "escape" the quotes (this is probably the Google search term you wanted).

$ ./time_it /xxx/blastp -outfmt \"6 std score\" -query x.faa -db nr -task blastp -out x.tsv

Thereby you program is correct, just the way you use bash to call a bash script changes.

ADD COMMENTlink written 2.1 years ago by Julian180
2

Oops, I didn't see you had answered this way already. Deleting my identical answer. This is standard quote behavior in bash.

ADD REPLYlink written 2.1 years ago by Aaronquinlan7.3k

escaping does not seem to work at least on my version of bash - although I agree that it should work and was my first guess as well -

ADD REPLYlink written 2.1 years ago by Istvan Albert ♦♦ 39k

I don't want to change the calling code, so no, that doesn't help. But thanks.

ADD REPLYlink written 2.1 years ago by Peter3.8k
Please log in to add an answer.

Help
Access
  • RSS
  • Stats
  • API

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.0.0
Traffic: 353 users visited in the last hour