Question: Getting a bash pipeline to output non-zero exit code when a step fails
1
gravatar for SaltedPork
21 months ago by
SaltedPork100
SaltedPork100 wrote:

I have a pipeline which looks something like below, its written in bash

Trimmomatic --> Samtools --> Smalt --> Lastz --> custom perl programs.

If anyone of those processes fails they usually return a non-0 exit code, usually a 1. However, my pipeline program will continue to run, report the error and return an exit code of 0, indicating success.

How can I get the pipeline to quit/stop with a non-0 exit code when a part of the pipeline fails?

command-line sh bash • 4.2k views
ADD COMMENTlink modified 21 months ago by bernatgel2.2k • written 21 months ago by SaltedPork100
2

Be aware that some software can "fail" when they don't fail.

ADD REPLYlink modified 21 months ago • written 21 months ago by h.mon28k
14
gravatar for RamRS
21 months ago by
RamRS24k
Houston, TX
RamRS24k wrote:

You're looking for set -eo pipefail. Add that after the first (#!/bin/bash) line and your script will fail if any part of your script fails.

See for more information: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/

ADD COMMENTlink modified 21 months ago • written 21 months ago by RamRS24k
1

Much better solution than mine!

ADD REPLYlink written 21 months ago by Joe15k

Thanks, this is a really good solution. Do you have any advice for if I wanted to just have the pipeline fail at certain points in the pipeline. Perhaps put a command in a loop and check that echo $? returns non-zero or not?

ADD REPLYlink written 21 months ago by SaltedPork100
2

If you follow Ram's link, you can combine it with the logic of mine below but invert it. i,e. if you have pipefail set, you can just invert the non-zero exit codes by doing something like:

mycommand || true

This means even if the command fails, the parent shell process will interpret is as having worked correctly (0), so you can use this to selectively override certain sections.

ADD REPLYlink written 21 months ago by Joe15k
2

Exactly. This works if the minority of statements are non-sensitive where OP wants to ignore the exit code. If the majority are non-sensitive, we're better off isolating the sensitive to separate scripts.

ADD REPLYlink written 21 months ago by RamRS24k
1

Yep. You'll need to check that. It will take a few trials though, as this is almost an implementation of try-catch blocks.

You could also place the sensitive parts in a separate script that has set -eo pipefail enabled and call it from your non-sensitive script.

ADD REPLYlink written 21 months ago by RamRS24k
4
gravatar for kloetzl
21 months ago by
kloetzl1.1k
European Union
kloetzl1.1k wrote:

To make the whole pipeline fail when any of the parts fail set the pipefail option. In zsh that can be done via setopt pipefail (don't know the bash equivalent by heart)/

ADD COMMENTlink written 21 months ago by kloetzl1.1k
1

bash equivalent would be set -o pipefail

ADD REPLYlink written 21 months ago by RamRS24k
3
gravatar for bernatgel
21 months ago by
bernatgel2.2k
Barcelona, Spain
bernatgel2.2k wrote:

I like the explicit error management described here, with a try, yell and die functions. As a bonus, you can modify die to do some additional logging of errors before actually dying and returning. It might be useful in addition to pipefail

yell() { echo "$0: $*" >&2; }
die() { yell "$*"; exit 111; }
try() { "$@" || die "cannot $*"; }
ADD COMMENTlink written 21 months ago by bernatgel2.2k
2
gravatar for Joe
21 months ago by
Joe15k
United Kingdom
Joe15k wrote:

mycommand || exit 1

Should work I think. Basically saying, either mycommand runs successfully to completion or exit the script with a non-zero status.

ADD COMMENTlink modified 21 months ago • written 21 months ago by Joe15k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1296 users visited in the last hour