Why Bash Scripts Fail Silently

Most of us have shipped a deployment script that ran fine on the laptop but exploded in CI because a single command returned a non‑zero exit code and the script kept marching on. Bash’s default behaviour is forgiving — it ignores failures, treats unset variables as empty strings, and lets a broken pipeline succeed if the last command exits cleanly. That leniency is great for interactive sessions, but it is a liability in automation.

The Minimal Safety Net

Adding three options at the top of every script changes the game:

#!/usr/bin/env bash
set -euo pipefail
# -e  Exit immediately if a command exits with a non‑zero status
# -u  Treat unset variables as an error when substituting
# -o pipefail  Return the exit status of the last command in a pipeline that failed

With set -e the script stops the moment something goes wrong. set -u catches typos like $DEPLOY_DIR vs $DEPLOY_DIRR before they wipe the wrong folder. pipefail ensures a failing grep inside a curl | grep | awk chain doesn’t get masked by the succeeding awk.

Graceful Cleanup with trap

Stopping early is only half the battle; you also need to release locks, remove temporary files, or notify monitoring. trap lets you register a function that runs on exit, error, or signals.

cleanup() {
    local exit_code=$?
    log "Script exiting with status $exit_code"
    # Remove temporary work directory
    [[ -d "$TMPDIR" ]] && rm -rf "$TMPDIR"
    # Release a flock lock if we used one
    [[ -n "$LOCK_FD" ]] && exec {LOCK_FD}>&-
    exit $exit_code
}
trap cleanup EXIT ERR INT TERM

The cleanup function captures the current exit code, logs it, and performs deterministic teardown. Because the trap fires on EXIT as well as ERR, normal termination and abrupt failures share the same path.

Real‑World Scenario: Blue‑Green Deploy

Imagine a blue‑green deployment script that swaps an Nginx upstream block, runs smoke tests, and then flips a symlink. Without strict mode, a failed sed that mangles the config would still execute the symlink swap, routing traffic to a broken backend. With the safety net in place:

  • The sed failure aborts the script instantly.
  • The trap removes the temporary config file and releases the deploy lock.
  • CI reports a clear non‑zero exit code, triggering a rollback pipeline.
Tip: Keep the trap function tiny and side‑effect free. Complex logic in a trap runs under constrained conditions (e.g., after a SIGKILL it won’t run at all).

Putting It All Together

Below is a production‑ready skeleton I drop into every new repo. It includes logging, a lock to prevent concurrent runs, and the strict‑mode + trap combo.

#!/usr/bin/env bash
set -euo pipefail

# ---------- configuration ----------
LOCK_FILE="/var/lock/deploy-${PROJECT_NAME}.lock"
TMPDIR="$(mktemp -d)"
LOG_FILE="${TMPDIR}/deploy.log"

# ---------- helpers ----------
log() { echo "[$(date -Is)] $*" | tee -a "$LOG_FILE"; }

die() { log "ERROR: $*"; exit 1; }

# ---------- lock ----------
exec {LOCK_FD}>&"$LOCK_FILE" || die "Cannot open lock file"
flock -n "$LOCK_FD" || die "Another deploy is running"

# ---------- trap ----------
cleanup() {
    local ec=$?
    log "Cleanup (exit $ec)"
    [[ -d "$TMPDIR" ]] && rm -rf "$TMPDIR"
    exec {LOCK_FD}>&-
    exit $ec
}
trap cleanup EXIT ERR INT TERM

# ---------- main logic ----------
log "Starting deploy for $PROJECT_NAME"
# ... fetch artefacts, render config, run tests ...
log "Deploy completed successfully"

What You Gain

  1. Predictable failures – the script stops at the first error, no silent corruption.
  2. Deterministic cleanup – temporary artefacts and locks disappear even on crash.
  3. Readable CI logs – every run ends with a clear exit code and a timestamped trace.

Adopting this pattern has saved me countless late‑night debugging sessions. It’s a small upfront investment that pays off every time a pipeline runs unattended.