Getting a bash pipeline to output non-zero exit code when a step fails
4
1
Entering edit mode
6.3 years ago
SaltedPork ▴ 170

I have a pipeline which looks something like below, its written in bash

Trimmomatic --> Samtools --> Smalt --> Lastz --> custom perl programs.

If anyone of those processes fails they usually return a non-0 exit code, usually a 1. However, my pipeline program will continue to run, report the error and return an exit code of 0, indicating success.

How can I get the pipeline to quit/stop with a non-0 exit code when a part of the pipeline fails?

bash command-line sh • 11k views
ADD COMMENT
2
Entering edit mode
ADD REPLY
14
Entering edit mode
6.3 years ago
Ram 44k

You're looking for set -eo pipefail. Add that after the first (#!/bin/bash) line and your script will fail if any part of your script fails.

See for more information: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/

ADD COMMENT
1
Entering edit mode

Much better solution than mine!

ADD REPLY
0
Entering edit mode

Thanks, this is a really good solution. Do you have any advice for if I wanted to just have the pipeline fail at certain points in the pipeline. Perhaps put a command in a loop and check that echo $? returns non-zero or not?

ADD REPLY
2
Entering edit mode

If you follow Ram's link, you can combine it with the logic of mine below but invert it. i,e. if you have pipefail set, you can just invert the non-zero exit codes by doing something like:

mycommand || true

This means even if the command fails, the parent shell process will interpret is as having worked correctly (0), so you can use this to selectively override certain sections.

ADD REPLY
2
Entering edit mode

Exactly. This works if the minority of statements are non-sensitive where OP wants to ignore the exit code. If the majority are non-sensitive, we're better off isolating the sensitive to separate scripts.

ADD REPLY
1
Entering edit mode

Yep. You'll need to check that. It will take a few trials though, as this is almost an implementation of try-catch blocks.

You could also place the sensitive parts in a separate script that has set -eo pipefail enabled and call it from your non-sensitive script.

ADD REPLY
4
Entering edit mode
6.3 years ago
kloetzl ★ 1.1k

To make the whole pipeline fail when any of the parts fail set the pipefail option. In zsh that can be done via setopt pipefail (don't know the bash equivalent by heart)/

ADD COMMENT
1
Entering edit mode

bash equivalent would be set -o pipefail

ADD REPLY
3
Entering edit mode
6.3 years ago
bernatgel ★ 3.4k

I like the explicit error management described here, with a try, yell and die functions. As a bonus, you can modify die to do some additional logging of errors before actually dying and returning. It might be useful in addition to pipefail

yell() { echo "$0: $*" >&2; }
die() { yell "$*"; exit 111; }
try() { "$@" || die "cannot $*"; }
ADD COMMENT
2
Entering edit mode
6.3 years ago
Joe 21k

mycommand || exit 1

Should work I think. Basically saying, either mycommand runs successfully to completion or exit the script with a non-zero status.

ADD COMMENT

Login before adding your answer.

Traffic: 2794 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6