Also see: pure bash bible (
A collection of pure POSIX sh alternatives to external processes.
The goal of this book is to document commonly-known and lesser-known methods of doing various tasks using only built-in POSIX sh
features. Using the snippets from this bible can help remove unneeded dependencies from scripts and in most cases make them faster. I came across these tips and discovered a few while developing KISS Linux and other smaller projects.
The snippets below are all linted using shellcheck
.
See something incorrectly described, buggy or outright wrong? Open an issue or send a pull request. If the bible is missing something, open an issue and a solution will be found.
- STRINGS
- Strip pattern from start of string
- Strip pattern from end of string
- Trim leading and trailing white-space from string
- Trim all white-space from string and truncate spaces
- Check if string contains a sub-string
- Check if string starts with sub-string
- Check if string ends with sub-string
- Split a string on a delimiter
- Trim quotes from a string
- FILES
- FILE PATHS
- LOOPS
- VARIABLES
- ESCAPE SEQUENCES
- PARAMETER EXPANSION
- CONDITIONAL EXPRESSIONS
- ARITHMETIC OPERATORS
- ARITHMETIC
- TRAPS
- OBSOLETE SYNTAX
- INTERNAL AND ENVIRONMENT VARIABLES
- AFTERWORD
Strip pattern from start of string
Example Function:
lstrip() { # Usage: lstrip "string" "pattern" printf '%sn' "${1##$2}" }
Example Usage:
$ lstrip "The Quick Brown Fox" "The " Quick Brown Fox
Strip pattern from end of string
Example Function:
rstrip() { # Usage: rstrip "string" "pattern" printf '%sn' "${1%%$2}" }
Example Usage:
$ rstrip "The Quick Brown Fox" " Fox" The Quick Brown
Trim leading and trailing white-space from string
This is an alternative to sed
, awk
, perl
and other tools. The
function below works by finding all leading and trailing white-space and
removing it from the start and end of the string.
Example Function:
trim_string() { # Usage: trim_string " example string " # Remove all leading white-space. # '${1%%[![:space:]]*}': Strip everything but leading white-space. # '${1#${XXX}}': Remove the white-space from the start of the string. trim=${1#${1%%[![:space:]]*}} # Remove all trailing white-space. # '${trim##*[![:space:]]}': Strip everything but trailing white-space. # '${trim%${XXX}}': Remove the white-space from the end of the string. trim=${trim%${trim##*[![:space:]]}} printf '%sn' "$trim" }
Example Usage:
$ trim_string " Hello, World " Hello, World $ name=" John Black " $ trim_string "$name" John Black
Trim all white-space from string and truncate spaces
This is an alternative to sed
, awk
, perl
and other tools. The
function below works by abusing word splitting to create a new string
without leading/trailing white-space and with truncated spaces.
Example Function:
set — $*
# Print the argument list as a string.
printf ‘%sn’ “$*”
# Re-enable globbing.
set +f
}” dir=”auto”>
# shellcheck disable=SC2086,SC2048 trim_all() { # Usage: trim_all " example string " # Disable globbing to make the word-splitting below safe. set -f # Set the argument list to the word-splitted string. # This removes all leading/trailing white-space and reduces # all instances of multiple spaces to a single (" " -> " "). set -- $* # Print the argument list as a string. printf '%sn' "$*" # Re-enable globbing. set +f }
Example Usage:
$ trim_all " Hello, World " Hello, World $ name=" John Black is my name. " $ trim_all "$name" John Black is my name.
Check if string contains a sub-string
Using a case statement:
case $var in *sub_string1*) # Do stuff ;; *sub_string2*) # Do other stuff ;; *) # Else ;; esac
Check if string starts with sub-string
Using a case statement:
case $var in sub_string1*) # Do stuff ;; sub_string2*) # Do other stuff ;; *) # Else ;; esac
Check if string ends with sub-string
Using a case statement:
case $var in *sub_string1) # Do stuff ;; *sub_string2) # Do other stuff ;; *) # Else ;; esac
Split a string on a delimiter
This is an alternative to cut
, awk
and other tools.
Example Function:
split() { # Disable globbing. # This ensures that the word-splitting is safe. set -f # Store the current value of 'IFS' so we # can restore it later. old_ifs=$IFS # Change the field separator to what we're # splitting on. IFS=$2 # Create an argument list splitting at each # occurance of '$2'. # # This is safe to disable as it just warns against # word-splitting which is the behavior we expect. # shellcheck disable=2086 set -- $1 # Print each list value on its own line. printf '%sn' "$@" # Restore the value of 'IFS'. IFS=$old_ifs # Re-enable globbing. set +f }
Example Usage:
$ split "apples,oranges,pears,grapes" "," apples oranges pears grapes $ split "1, 2, 3, 4, 5" ", " 1 2 3 4 5
Trim quotes from a string
Example Function:
trim_quotes() { # Usage: trim_quotes "string" # Disable globbing. # This makes the word-splitting below safe. set -f # Store the current value of 'IFS' so we # can restore it later. old_ifs=$IFS # Set 'IFS' to ["']. IFS="' # Create an argument list, splitting the # string at ["']. # # Disable this shellcheck error as it only # warns about word-splitting which we expect. # shellcheck disable=2086 set -- $1 # Set 'IFS' to blank to remove spaces left # by the removal of ["']. IFS= # Print the quote-less string. printf '%sn' "$*" # Restore the value of 'IFS'. IFS=$old_ifs # Re-enable globbing. set +f }
Example Usage:
$ var="'Hello', "World"" $ trim_quotes "$var" Hello, World
Parsing a key=val
file.
This could be used to parse a simple key=value
configuration file.
# printf ‘warning %s is not a valid variable namen’ “$key”
done < "file"" dir="auto">
# Setting 'IFS' tells 'read' where to split the string. while IFS='=' read -r key val; do # Skip over lines containing comments. # (Lines starting with '#'). [ "${key###*}" ] || continue # '$key' stores the key. # '$val' stores the value. printf '%s: %sn' "$key" "$val" # Alternatively replacing 'printf' with the following # populates variables called '$key' with the value of '$val'. # # NOTE: I would extend this with a check to ensure 'key' is # a valid variable name. # export "$key=$val" # # Example with error handling: # export "$key=$val" 2>/dev/null || # printf 'warning %s is not a valid variable namen' "$key" done < "file"
Get the first N lines of a file
Alternative to the head
command.
Example Function:
head() { # Usage: head "n" "file" while IFS= read -r line; do printf '%sn' "$line" i=$((i+1)) [ "$i" = "$1" ] && return done < "$2" # 'read' used in a loop will skip over # the last line of a file if it does not contain # a newline and instead contains EOF. # # The final line iteration is skipped as 'read' # exits with '1' when it hits EOF. 'read' however, # still populates the variable. # # This ensures that the final line is always printed # if applicable. [ -n "$line" ] && printf %s "$line" }
Example Usage:
$ head 2 ~/.bashrc # Prompt PS1='➜ ' $ head 1 ~/.bashrc # Prompt
Get the number of lines in a file
Alternative to wc -l
.
Example Function:
lines() { # Usage: lines "file" # '|| [ -n "$line" ]': This ensures that lines # ending with EOL instead of a newline are still # operated on in the loop. # # 'read' exits with '1' when it sees EOL and # without the added test, the line isn't sent # to the loop. while IFS= read -r line || [ -n "$line" ]; do lines=$((lines+1)) done < "$1" printf '%sn' "$lines" }
Example Usage:
Count files or directories in directory
This works by passing the output of the glob to the function and then counting the number of arguments.
Example Function:
count() { # Usage: count /path/to/dir/* # count /path/to/dir/*/ [ -e "$1" ] && printf '%sn' "$#" || printf '%sn' 0 }
Example Usage:
# Count all files in dir. $ count ~/Downloads/* 232 # Count all dirs in dir. $ count ~/Downloads/*/ 45 # Count all jpg files in dir. $ count ~/Pictures/*.jpg 64
Create an empty file
Alternative to touch
.
Alternative to the dirname
command.
Example Function:
dirname() { # Usage: dirname "path" # If '$1' is empty set 'dir' to '.', else '$1'. dir=${1:-.} # Strip all trailing forward-slashes '/' from # the end of the string. # # "${dir##*[!/]}": Remove all non-forward-slashes # from the start of the string, leaving us with only # the trailing slashes. # "${dir%%"${}"}": Remove the result of the above # substitution (a string of forward slashes) from the # end of the original string. dir=${dir%%"${dir##*[!/]}"} # If the variable *does not* contain any forward slashes # set its value to '.'. [ "${dir##*/*}" ] && dir=. # Remove everything *after* the last forward-slash '/'. dir=${dir%/*} # Again, strip all trailing forward-slashes '/' from # the end of the string (see above). dir=${dir%%"${dir##*[!/]}"} # Print the resulting string and if it is empty, # print '/'. printf '%sn' "${dir:-/}" }
Example Usage:
$ dirname ~/Pictures/Wallpapers/1.jpg /home/black/Pictures/Wallpapers/ $ dirname ~/Pictures/Downloads/ /home/black/Pictures/
Get the base-name of a file path
Alternative to the basename
command.
Example Function:
basename() { # Usage: basename "path" ["suffix"] # Strip all trailing forward-slashes '/' from # the end of the string. # # "${1##*[!/]}": Remove all non-forward-slashes # from the start of the string, leaving us with only # the trailing slashes. # "${1%%"${}"}: Remove the result of the above # substitution (a string of forward slashes) from the # end of the original string. dir=${1%${1##*[!/]}} # Remove everything before the final forward-slash '/'. dir=${dir##*/} # If a suffix was passed to the function, remove it from # the end of the resulting string. dir=${dir%"$2"} # Print the resulting string and if it is empty, # print '/'. printf '%sn' "${dir:-/}" }
Example Usage:
$ basename ~/Pictures/Wallpapers/1.jpg 1.jpg $ basename ~/Pictures/Wallpapers/1.jpg .jpg 1 $ basename ~/Pictures/Downloads/ Downloads
Loop over a (small) range of numbers
Alternative to seq
and only suitable for small and static number ranges. The number list can also be replaced with a list of words, variables etc.
# Loop from 0-10. for i in 0 1 2 3 4 5 6 7 8 9 10; do printf '%sn' "$i" done
Loop over a variable range of numbers
Alternative to seq
.
# Loop from var-var. start=0 end=50 while [ "$start" -le "$end" ]; do printf '%sn' "$start" start=$((start+1)) done
Loop over the contents of a file
while IFS= read -r line || [ -n "$line" ]; do printf '%sn' "$line" done < "file"
Loop over files and directories
Don’t use ls
.
CAVEAT: When the glob does not match anything (empty directory or no matching files) the variable will contain the unexpanded glob. To avoid working on unexpanded globs check the existence of the file contained in the variable using the appropriate file conditional. Be aware that symbolic links are resolved.
# Greedy example. for file in *; do [ -e "$file" ] || [ -L "$file" ] || continue printf '%sn' "$file" done # PNG files in dir. for file in ~/Pictures/*.png; do [ -f "$file" ] || continue printf '%sn' "$file" done # Iterate over directories. for dir in ~/Downloads/*/; do [ -d "$dir" ] || continue printf '%sn' "$dir" done
Name and access a variable based on another variable
$ var="world" $ eval "hello_$var=value" $ eval printf '%sn' "$hello_$var" value
Contrary to popular belief, there is no issue in utilizing raw escape sequences. Using tput
abstracts the same ANSI sequences as if printed manually. Worse still, tput
is not actually portable. There are a number of tput
variants each with different commands and syntaxes (try tput setaf 3
on a FreeBSD system). Raw sequences are fine.
Text Colors
NOTE: Sequences requiring RGB values only work in True-Color Terminal Emulators.
Sequence | What does it do? | Value |
---|---|---|
|