asking for optimized bash scripting

Question

[QX]

Hi all,

I usually organized my analysis by make bash scripts files that import a general config.sh file like this:

project=../projects/test/
raw_dir=../projects/fastq/
script_dir=${project}scripts/ 
log_dir=${project}logs/
mkdir -p $log_dir    

# ---fastqc check
fastqc_dir=${project}fastqc/
fastqc_untrimmed_dir=${fastqc_dir}/fastqc_untrimmed/
demux_dir=${project}fastq/demux/
demux_trim_dir=${project}fastq/demux_trim/
fastqc_demux_dir=${fastqc_dir}/demux/

but it look kind of ugly and not easily reading.

Is there any other way to set a global variabel bash or yaml file that can be looked better?

[Joe]

One thing that definitely could break here @OP is that you need to quote your variables.

e.g. fastqc_demux_dir=${fastqc_dir}/demux/ should be fastqc_demux_dir="${fastqc_dir}/demux/"

Stops weird things happening if there are special characters or whitespace in your filenames.

Answer 1

[Joe]

You can set global ("environment" variables) in bash, but they aren't generally the right way to do this.

The "right" way to do this in my view would be to simply pass the folders as an argument to the script call (e.g. bash scriptname.sh folder1 option1 folder2 option2 - or better yet, use proper argument parsing and create scripts with actual arguments e.g. bash scriptname.sh --directory folder1 --output folder2 etc).

Your general approach of defining a core output/project folder is the right one, and the script can handle making subfolders internally to segregate outputs if you want.

Here's an example:

https://github.com/jrjhealey/SysAdmin/blob/master/sync_data.sh

Answer 2

[cmdcolin]

the script you wrote is fairly small so there's not that much I would do to change it, but AI coding helpers are pretty good writing a lot of languages including bash. You can ask the coding helper to "improve the readability and re-usability of this script" and it will reorganize things and try to help improve the format

it's not always perfect, but can be a good start.

example

#!/bin/bash

# Project Directory Setup Script
# Usage: ./setup_project.sh [project_dir] [raw_fastq_dir]

set -euo pipefail

# Configuration
PROJECT_ROOT=$1
RAW_FASTQ_DIR=$2

# Project directories
SCRIPT_DIR="${PROJECT_ROOT}/scripts"
LOG_DIR="${PROJECT_ROOT}/logs"
FASTQC_DIR="${PROJECT_ROOT}/fastqc"
DEMUX_DIR="${PROJECT_ROOT}/fastq/demux"
DEMUX_TRIM_DIR="${PROJECT_ROOT}/fastq/demux_trim"
FASTQC_UNTRIMMED_DIR="${FASTQC_DIR}/fastqc_untrimmed"
FASTQC_DEMUX_DIR="${FASTQC_DIR}/demux"

echo "Setting up project: $PROJECT_ROOT"
mkdir -p "$SCRIPT_DIR" "$LOG_DIR" "$FASTQC_UNTRIMMED_DIR" "$DEMUX_DIR" "$DEMUX_TRIM_DIR" "$FASTQC_DEMUX_DIR"