Shell_expansion

Chapter 27: Shell Expansion in Bash

Overview

Shell expansion is the process by which the shell interprets special characters and replaces them with their values before executing a command. Understanding expansion is crucial for writing effective bash scripts and understanding how the shell processes your commands.

---

Types of Expansion

┌────────────────────────────────────────────────────────────────┐
│                   Shell Expansion Types                           │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│   1. Brace Expansion      {a,b,c} → a b c                    │
│   2. Tilde Expansion      ~ → /home/user                     │
│   3. Parameter Expansion   $VAR → value                        │
│   4. Command Substitution $(cmd) → output                    │
│   5. Arithmetic Expansion $((expr)) → result                 │
│   6. Word Splitting       splits on IFS                      │
│   7. Pathname Expansion   *.txt → file1.txt file2.txt       │
│   8. Quote Removal        "text" → text                      │
│                                                                │
│   Order of Operations:                                          │
│   ──────────────────                                           │
│   1. Brace Expansion                                            │
│   2. Tilde Expansion                                           │
│   3. Parameter, Command, Arithmetic Expansion                 │
│   4. Word Splitting                                            │
│   5. Pathname Expansion                                        │
│   6. Quote Removal                                             │
│                                                                │
└────────────────────────────────────────────────────────────────┘

---

Brace Expansion

Basic Braces

# Generate sequences
echo {1..5}
# 1 2 3 4 5

echo {01..05}
# 01 02 03 04 05

echo {a..e}
# a b c d e

echo {A..E}
# A B C D E

Combinations

# File extensions
echo file.{txt,log,json}
# file.txt file.log file.json

# Directory structure
mkdir -p project/{src,tests,docs}

# Multiple levels
cp file.{log,bak}
# file.log file.bak

# Nested braces
echo {{A,B},{C,D}}
# A B C D

Padding with Zeros

# With leading zeros
echo {01..10}
# 01 02 03 04 05 06 07 08 09 10

# Without zeros
echo {1..10}
# 1 2 3 4 5 6 7 8 9 10

---

Tilde Expansion

Basic Tilde

/home/username

# Home directory
echo ~
# Specific user's home
echo ~root
# /root

echo ~guest
# /home/guest

Tilde in Paths

# Current user's home
ls ~/documents

# Relative to home
ls ~-       # Previous directory ($OLDPWD)
echo ~+     # Current directory ($PWD)

# In variable
export PROJECT_DIR=~/projects/myapp

---

Parameter Expansion

Basic Variables

# Simple expansion
echo $HOME
echo $USER
echo $PATH

# Curly braces (recommended)
echo ${HOME}
echo ${USER}

Advanced Parameter Expansion

# Default values
${parameter:-word}     # Use default if unset or null
${parameter:=word}    # Assign default if unset or null
${parameter:+word}    # Use alternate if set
${parameter:?word}    # Error if unset or null
${parameter:?}        # Print error if unset

# Examples
echo ${HOME:-/tmp}           # Use /tmp if HOME unset
echo ${HOME:=/home/default}  # Assign if unset
echo ${HOME:+/alternate}    # Use alternate if set

# Length
${#parameter}               # Length of value
echo ${#HOME}

String Operations

# Substring removal
${parameter#pattern}    # Remove shortest match from beginning
${parameter##pattern}  # Remove longest match from beginning
${parameter%pattern}   # Remove shortest match from end
${parameter%%pattern} # Remove longest match from end

# Examples
path="/home/user/documents/file.txt"
echo ${path##*/}        # file.txt (remove path)
echo ${path%.txt}       # /home/user/documents/file (remove extension)

# Substring
${parameter:offset}           # From offset to end
${parameter:offset:length}   # Slice

text="Hello World"
echo ${text:0:5}     # Hello
echo ${text:6:5}     # World

Pattern Replacement

# Replace first match
${parameter/pattern/string}

# Replace all matches
${parameter//pattern/string}

# Examples
text="hello world hello"
echo ${text/hello/bye}       # bye world hello
echo ${text//hello/bye}     # bye world bye

# Prefix/suffix
${parameter/#pattern/string}   # Replace if at beginning
${parameter/%pattern/string}  # Replace if at end

file="test.txt"
echo ${file/#test/backup}   # backup.txt
echo ${file/%txt/log}       # test.log

---

Command Substitution

Basic Command Substitution

# Modern syntax (recommended)
$(command)

# Legacy syntax
`command`

# Examples
current_date=$(date)
echo $current_date

files=$(ls -1)
echo $files

# Using output in variable
count=$(ls -1 *.txt | wc -l)
echo "Text files: $count"

Nested Command Substitution

# Nested commands
outer=$(inner=$(echo hello); echo $inner)
echo $outer

# Practical nested
files=$(ls $(pwd))

---

Arithmetic Expansion

Basic Math

# Using $(( ))
result=$(( 5 + 3 ))
echo $result  # 8

# Various operations
echo $(( 10 - 3 ))    # 7
echo $(( 5 * 3 ))     # 15
echo $(( 10 / 3 ))    # 3 (integer)
echo $(( 10 % 3 ))    # 1

# Power
echo $(( 2 ** 10 ))   # 1024

Variables in Math

x=10
y=3

echo $(( x + y ))     # 13
echo $(( x - y ))     # 7
echo $(( x * y ))     # 30
echo $(( x / y ))     # 3
echo $(( x % y ))     # 1

Increment/Decrement

x=5

# Pre-increment
echo $(( ++x ))    # 6
echo $x             # 6

# Post-increment
echo $(( x++ ))    # 6
echo $x             # 7

# Combined
x=5
x=$(( x + 1 ))
echo $x            # 6

Comparison

# Returns 1 (false) or 0 (true)
echo $(( 5 > 3 ))   # 1
echo $(( 5 < 3 ))   # 0
echo $(( 5 == 5 ))  # 1
echo $(( 5 != 3 ))  # 1

# In conditions
if (( 5 > 3 )); then
    echo "True"
fi

---

Word Splitting

IFS (Internal Field Separator)

# Default: space, tab, newline
text="one two three"
for word in $text; do
    echo "$word"
done

# Custom IFS
text="one:two:three"
IFS=':' read -ra words <<< "$text"
echo "${words[0]}"  # one

Splitting Rules

# Word splitting after expansion
var="hello world"
echo $var    # hello world (split into two)
echo "$var"  # hello world (single)

# Multiple spaces
var="one   two    three"
echo $var    # one two three
echo "$var"  # one   three spaces

---

Pathname Expansion

Wildcards

# * - Any characters
ls *.txt

# ? - Single character
ls file?.txt

# [] - Character class
ls file[0-9].txt
ls file[a-z].txt

# Extended globbing (shopt -s extglob)
ls @(foo|bar).txt     # foo or bar
ls !(foo).txt         # anything except foo
ls *(foo).txt         # zero or more
ls +(foo).txt         # one or more
ls?(foo).txt          # zero or one

---

Quote Removal

Types of Quotes

# Double quotes - allow expansion
echo "Hello $USER"      # Hello username

# Single quotes - literal
echo 'Hello $USER'      # Hello $USER

# Backslash - escape
echo "Hello \$USER"     # Hello $USER
echo "Path: \$HOME"     # Path: /home/user

---

Practical Examples

File Operations

# Create multiple files
touch file{1..5}.txt

# Backup files
cp file.txt{,.bak}

# Create directories
mkdir -p project/{src,tests,docs}/{css,js}

# Remove extensions
for f in *.txt; do
    mv "$f" "${f%.txt}"
done

Date/Time in Filenames

# Current date
filename="backup_$(date +%Y%m%d).tar.gz"
echo $filename

# With time
timestamp=$(date +%Y%m%d_%H%M%S)
logfile="app_$timestamp.log"

Conditional Variables

# Use default if not set
CONFIG_DIR=${CONFIG_DIR:-/etc/myapp}
export CONFIG_DIR

# Set default if not set
DATABASE_URL=${DATABASE_URL:=postgresql://localhost:5432/db}

---

Debugging Expansions

Using set -x

#!/usr/bin/env bash
# Enable expansion tracing

set -x

var="hello"
echo "$var"

result=$(ls)
echo "$result"

set +x

Using echo for Debugging

# Show what expands to
echo "Expansion: $var"
echo "Arithmetic: $(( 5 + 3 ))"

# Check what glob expands to
echo *.txt

---

Order of Operations

┌────────────────────────────────────────────────────────────────┐
│              Order of Shell Expansions                           │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│   Input:  echo {a,b}$VAR$(date)*.txt                          │
│                                                                │
│   Step 1: Brace Expansion                                      │
│   ─────────────────────────────────────                        │
│   echo a$VAR$(date)*.txt b$VAR$(date)*.txt                    │
│                                                                │
│   Step 2: Tilde Expansion                                      │
│   ─────────────────────────────────────                        │
│   (if ~ present)                                               │
│                                                                │
│   Step 3: Parameter, Command, Arithmetic Expansion              │
│   ─────────────────────────────────────                        │
│   echo avalue2024-01-01*.txt bvalue2024-01-01*.txt            │
│                                                                │
│   Step 4: Word Splitting                                       │
│   ─────────────────────────────────────                        │
│   (if unquoted)                                                │
│                                                                │
│   Step 5: Pathname Expansion                                   │
│   ─────────────────────────────────────                        │
│   echo avalue2024-01-01file1.txt file2.txt                     │
│            bvalue2024-01-01file1.txt file2.txt                │
│                                                                │
│   Step 6: Quote Removal                                        │
│   ─────────────────────────────────────                        │
│   (quotes removed)                                             │
│                                                                │
└────────────────────────────────────────────────────────────────┘

---

Summary

In this chapter, you learned:

• ✅ Types of shell expansion
• ✅ Brace expansion for generating sequences
• ✅ Tilde expansion for home directories
• ✅ Parameter expansion with advanced features
• ✅ Command substitution
• ✅ Arithmetic expansion
• ✅ Word splitting and IFS
• ✅ Pathname expansion (globbing)
• ✅ Quote removal
• ✅ Practical examples
• ✅ Debugging expansions

---

Next Steps

Continue to the next chapter to learn about Parameter Expansion in detail.

---

Previous Chapter: Environment Variables Next Chapter: Parameter Expansion