Blog:Beware the differences between /bin/sh and /bin/bash

Question

[Ram]

Hello everyone,

This is slightly off topic, but something I stumbled upon quite recently. I'm sure veteran unix programmers would have encountered this too.

I'll start off with something almost everyone knows - not all greps are equal. Not all cuts are the same. There might be certain options available to binaries in one flavor that are not available to their equivalent binaries in a different flavor. For example, cut --complement is available in GNU cut, but not in macOS default BSD's cut.

Yesterday, I discovered that shell are by themselves quite different too. It seems totally obvious right now, but when I encountered the bug, it puzzled me.

I was testing a shell script x.sh that has a statement:

python util.py -a <(cut -f2 input1.txt) -b input2.txt

Now, this uses Process Substitution (<(...)) - a construct I use very often. I've tested it on an interactive shell, I've used it in shell scripts, so I know it works.

Except, this time it didn't. Took me a while to zero in on this statement, but when I did, it shocked me. How does something so basic not work?

When I started learning Linux in 2005, I was told I could use the dot-slash (./) or the sh binary to execute a shell script. Turns out, I was using the syntax sh x.sh to execute the script, out of force of habit. And even though I had the interpreter in the first line of the script (#!/bin/bash), since I was explicitly specifying sh as the interpreter, it was passing my script to sh. The POSIX compliant sh, as it turns out, is not as rich as bash in features and cannot handle process substitution. So I had to ensure that bash was being used to execute the script, which I achieved by advising the human users not to use sh and also putting in a check for POSIX compliant execution.

A more detailed resource: http://www.faqs.org/faqs/unix-faq/shell/shell-differences/

Anyway, the point is, something we take for granted could prove quite daunting when it pulls the rug out from under our feet. I hope this information helps someone out someday!

Cheers!

--
Ram

[Jean-Karim Heriche]

This goes again to show that when programming, one needs to be precise. If you mean to execute a command, call this command, not something else that you think will do the same thing. This also applies to data, like when one assumes a particular format, does a lot of work to reach incorrect conclusions just because the wrong piece of information was processed. Bottom line: don't make assumptions or if you do, check that they are valid and report when they're not.

[Ram]

Agreed. Also, the more you learn, the more you question your basic assumptions. Because I was told sh shellScript.sh and ./shellScript.sh were equivalent from when I was a kid, I did not question that. Plus, it worked fine until yesterday - largely because I used them interchangeably and was fortunate (or unfortunate) enough to not stumble on the sh + <() combo.

I'm glad I made that error at last. These kind of assumption shattering insights happen very rarely :)

[Ram]

Turns out, Ubuntu (my go-to distro until 2014, when I started working on OS X+Homebrew) links sh to /bin/bash. Looks like I haven't yet graduated from the Ubuntu High School after all!

[kloetzl]

I feel that a lot of people use the terms command line, shell and bash interchangeably. As a fan of the zsh I wince when that happens.

[Ram]

Quick questions:

1. Why zsh? Any particular reason to prefer that over bash?
2. I know the difference between shell and bash, but I rarely use the term "command line" except when I'm referring to the UI type (CLI vs GUI). What, in your view, is a valid usage for "command line"?

[kloetzl]

1. zsh has superior autocompletion to bash. Also a lot of little things: popd, .., =(). Checkout biozsh for zsh greatness in bioinformatics.
2. I agree with you there.

[Ram]

I use pushd, popd and dirs with bash. I'll check out biozsh, thanks!