Blog: Beware the differences between /bin/sh and /bin/bash
gravatar for RamRS
3.4 years ago by
Baylor College of Medicine, Houston, TX
RamRS30k wrote:

Hello everyone,

This is slightly off topic, but something I stumbled upon quite recently. I'm sure veteran unix programmers would have encountered this too.

I'll start off with something almost everyone knows - not all greps are equal. Not all cuts are the same. There might be certain options available to binaries in one flavor that are not available to their equivalent binaries in a different flavor. For example, cut --complement is available in GNU cut, but not in macOS default BSD's cut.


Yesterday, I discovered that shell are by themselves quite different too. It seems totally obvious right now, but when I encountered the bug, it puzzled me.

I was testing a shell script that has a statement:

python -a <(cut -f2 input1.txt) -b input2.txt

Now, this uses Process Substitution (<(...)) - a construct I use very often. I've tested it on an interactive shell, I've used it in shell scripts, so I know it works.

Except, this time it didn't. Took me a while to zero in on this statement, but when I did, it shocked me. How does something so basic not work?

When I started learning Linux in 2005, I was told I could use the dot-slash (./) or the sh binary to execute a shell script. Turns out, I was using the syntax sh to execute the script, out of force of habit. And even though I had the interpreter in the first line of the script (#!/bin/bash), since I was explicitly specifying sh as the interpreter, it was passing my script to sh. The POSIX compliant sh, as it turns out, is not as rich as bash in features and cannot handle process substitution. So I had to ensure that bash was being used to execute the script, which I achieved by advising the human users not to use sh and also putting in a check for POSIX compliant execution.

A more detailed resource:

Anyway, the point is, something we take for granted could prove quite daunting when it pulls the rug out from under our feet. I hope this information helps someone out someday!


-- Ram

scripts shell unix blog • 5.6k views
ADD COMMENTlink written 3.4 years ago by RamRS30k

This goes again to show that when programming, one needs to be precise. If you mean to execute a command, call this command, not something else that you think will do the same thing. This also applies to data, like when one assumes a particular format, does a lot of work to reach incorrect conclusions just because the wrong piece of information was processed. Bottom line: don't make assumptions or if you do, check that they are valid and report when they're not.

ADD REPLYlink written 3.4 years ago by Jean-Karim Heriche23k

Agreed. Also, the more you learn, the more you question your basic assumptions. Because I was told sh and ./ were equivalent from when I was a kid, I did not question that. Plus, it worked fine until yesterday - largely because I used them interchangeably and was fortunate (or unfortunate) enough to not stumble on the sh + <() combo.

I'm glad I made that error at last. These kind of assumption shattering insights happen very rarely :)

ADD REPLYlink written 3.4 years ago by RamRS30k

Turns out, Ubuntu (my go-to distro until 2014, when I started working on OS X+Homebrew) links sh to /bin/bash. Looks like I haven't yet graduated from the Ubuntu High School after all!

ADD REPLYlink modified 3.4 years ago • written 3.4 years ago by RamRS30k

I feel that a lot of people use the terms command line, shell and bash interchangeably. As a fan of the zsh I wince when that happens.

ADD REPLYlink written 3.4 years ago by kloetzl1.1k

Quick questions:

  1. Why zsh? Any particular reason to prefer that over bash?
  2. I know the difference between shell and bash, but I rarely use the term "command line" except when I'm referring to the UI type (CLI vs GUI). What, in your view, is a valid usage for "command line"?
ADD REPLYlink written 3.4 years ago by RamRS30k
  1. zsh has superior autocompletion to bash. Also a lot of little things: popd, .., =(). Checkout biozsh for zsh greatness in bioinformatics.
  2. I agree with you there.
ADD REPLYlink written 3.4 years ago by kloetzl1.1k

I use pushd, popd and dirs with bash. I'll check out biozsh, thanks!

ADD REPLYlink written 3.4 years ago by RamRS30k
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1934 users visited in the last hour