Signals

Chapter 24: Signals and Traps

Overview

Signals are software interrupts that notify processes of events. The trap command allows scripts to respond to signals, making them essential for creating robust DevOps scripts that handle interrupts gracefully.

---

Understanding Signals

What are Signals?

Signals are messages sent to a running process to notify it of events. They can be triggered by:

• Hardware (keyboard, hardware errors)
• Software (program errors, conditions)
• Users (Ctrl+C, kill commands)
• System (resource limits, parent death)

┌────────────────────────────────────────────────────────────────┐
│                       Signal Flow                               │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│   Source          Signal          Process                     │
│   ──────         ───────         ───────                     │
│                                                                │
│   User ──────► Ctrl+C ──────► SIGINT ─────────► Interrupt    │
│   User ──────► kill ────────► SIGTERM ───────► Terminate    │
│   Kernel ─────► Hardware ─────► SIGSEGV ──────► Crash       │
│   System ─────► Timeout ──────► SIGALRM ──────► Alarm       │
│                                                                │
└────────────────────────────────────────────────────────────────┘

---

Common Signals

POSIX Signals

Signal	Number	Description	Default Action
SIGHUP	1	Hangup (terminal closed)	Terminate
SIGINT	2	Interrupt (Ctrl+C)	Terminate
SIGQUIT	3	Quit (Ctrl+)	Terminate + Core
SIGKILL	9	Kill (cannot be caught)	Terminate
SIGTERM	15	Termination (graceful)	Terminate
SIGUSR1	10	User-defined 1	Terminate
SIGUSR2	12	User-defined 2	Terminate
SIGSEGV	11	Segmentation fault	Terminate + Core
SIGALRM	14	Alarm clock	Terminate
SIGCHLD	17	Child stopped/exited	Ignore

Signal Names in Bash

# Using signal names (portable)
trap 'commands' SIGHUP SIGINT SIGTERM

# Using signal numbers
trap 'commands' 1 2 15

# Using SIG prefix (modern bash)
trap 'commands' SIGTERM
trap 'commands' EXIT

---

The trap Command

Basic Syntax

trap [OPTIONS] ['command(s)'] [SIGNALS]

# Options:
# -  : Default signal handling
# -l : List all signals
# -p : Print trap commands

Common Uses

# Run on EXIT (all exits)
trap 'echo "Script exited"' EXIT

# Run on specific signals
trap 'echo "Interrupted"' INT
trap 'echo "Killed"' TERM

# Ignore signal
trap '' INT  # Ignore Ctrl+C

# Default signal handling
trap - INT  # Reset to default

---

Practical Examples

1. Cleanup on Exit

#!/usr/bin/env bash
# Cleanup script

TEMP_DIR="/tmp/myapp_$$"
LOG_FILE="/var/log/myapp.log"

# Create temp directory
mkdir -p "$TEMP_DIR"

# Cleanup function
cleanup() {
    echo "$(date): Cleaning up..." >> "$LOG_FILE"
    rm -rf "$TEMP_DIR"
    echo "$(date): Cleanup complete" >> "$LOG_FILE"
}

# Run cleanup on EXIT, HUP, INT, TERM
trap cleanup EXIT HUP INT TERM

# Main script
echo "Starting application..."
sleep 10
echo "Application finished"

2. Ignore Interrupt During Critical Section

#!/usr/bin/env bash

echo "Starting critical operation..."

# Ignore Ctrl+C during critical section
trap '' INT

# Critical section
sleep 30

# Restore interrupt handling
trap - INT

echo "Critical section complete"

3. Graceful Shutdown

#!/usr/bin/env bash

PID_FILE="/var/run/myapp.pid"

# Start daemon
start() {
    echo "Starting service..."
    ./myapp.sh &
    echo $! > "$PID_FILE"

    # Set trap for shutdown
    trap 'stop' TERM INT
}

# Stop daemon
stop() {
    echo "Stopping service..."
    if [[ -f "$PID_FILE" ]]; then
        kill $(cat "$PID_FILE")
        rm "$PID_FILE"
    fi
}

# Cleanup
cleanup() {
    stop
    exit 0
}

trap cleanup EXIT
trap stop TERM INT

# Main
start

# Keep running
while true; do
    sleep 1
done

4. Retry with Timeout

#!/usr/bin/env bash

TIMEOUT=60
INTERVAL=5

retry_command() {
    local cmd="$1"
    local elapsed=0

    while [[ $elapsed -lt $TIMEOUT ]]; do
        if eval "$cmd"; then
            echo "Command succeeded"
            return 0
        fi
        echo "Command failed, retrying in $INTERVAL seconds..."
        sleep $INTERVAL
        ((elapsed += INTERVAL))
    done

    echo "Command failed after $TIMEOUT seconds"
    return 1
}

# Handle interrupt during retry
trap 'echo "Interrupted"; exit 130' INT TERM

retry_command "curl -sf http://localhost:8080/health"

---

Signal Handling in Scripts

Different Responses to Signals

#!/usr/bin/env bash

# Ignore signal
trap '' SIGTSTP  # Ignore Ctrl+Z

# Run function on signal
cleanup() {
    echo "Cleaning up..."
}

trap cleanup EXIT

# Multiple signals
trap 'echo "SIGHUP"' HUP
trap 'echo "SIGTERM"' TERM
trap 'echo "SIGINT"' INT

EXIT Trap with Status

#!/usr/bin/env bash

# Capture exit status
trap 'exit_func' EXIT

exit_func() {
    local exit_status=$?
    echo "Script exiting with status: $exit_status"

    if [[ $exit_status -ne 0 ]]; then
        echo "Error occurred, sending alert..."
    fi
}

# Your script logic
echo "Script running..."
exit 1  # Test exit

---

DEBUG and ERR Traps

DEBUG Trap

#!/usr/bin/env bash

# Run before each command
debug() {
    echo "Executing: $BASH_COMMAND"
}

trap debug DEBUG

# Commands will be logged
echo "Hello"
ls -la

ERR Trap

#!/usr/bin/env bash

# Run on command failure
error() {
    echo "Error on line $LINENO: $BASH_COMMAND"
    exit 1
}

set -E  # Enable ERR trap in functions
trap error ERR

# This will trigger error trap
false
echo "This won't print"

---

Real-World DevOps Examples

Database Connection Manager

#!/usr/bin/env bash
# Database backup with cleanup

DB_HOST="localhost"
DB_NAME="mydb"
BACKUP_DIR="/backups"

backup_database() {
    local timestamp=$(date +%Y%m%d_%H%M%S)
    local backup_file="$BACKUP_DIR/db_$timestamp.sql"

    echo "Starting backup..."

    if ! mysqldump -h "$DB_HOST" "$DB_NAME" > "$backup_file"; then
        echo "Backup failed"
        return 1
    fi

    echo "Backup complete: $backup_file"
    return 0
}

cleanup() {
    local exit_status=$?
    echo "Cleaning up..."

    # Kill any remaining connections
    # Remove temp files
    # Unlock resources

    echo "Exit status: $exit_status"
    exit $exit_status
}

trap cleanup EXIT

# Run backup
backup_database

Web Server Health Check

#!/usr/bin/env bash

URL="${1:-http://localhost:8080/health}"
CHECK_INTERVAL=10

check_health() {
    local response=$(curl -sf -o /dev/null -w "%{http_code}" "$URL")

    if [[ "$response" != "200" ]]; then
        echo "$(date): Health check failed - HTTP $response"
        return 1
    fi

    echo "$(date): Health check passed"
    return 0
}

# Graceful shutdown
shutdown() {
    echo "$(date): Shutdown signal received"
    exit 0
}

trap shutdown TERM INT

# Main loop
while true; do
    check_health || echo "Alert: Health check failed"
    sleep $CHECK_INTERVAL
done

Docker Cleanup Script

#!/usr/bin/env bash

# Stop and remove containers
cleanup_containers() {
    echo "Stopping containers..."
    docker stop $(docker ps -aq) 2>/dev/null

    echo "Removing containers..."
    docker rm $(docker ps -aq) 2>/dev/null
}

# Remove images
cleanup_images() {
    echo "Removing unused images..."
    docker image prune -f
}

# Remove volumes
cleanup_volumes() {
    echo "Removing unused volumes..."
    docker volume prune -f
}

# Full cleanup
full_cleanup() {
    cleanup_containers
    cleanup_images
    cleanup_volumes
    echo "Cleanup complete"
}

# Handle interrupts
trap 'echo "Interrupted"; exit 130' INT TERM
trap 'echo "Killed"; exit 137' KILL

# Run based on argument
case "${1:-all}" in
    containers)
        cleanup_containers
        ;;
    images)
        cleanup_images
        ;;
    volumes)
        cleanup_volumes
        ;;
    *)
        full_cleanup
        ;;
esac

---

Signal Safety

What Can Be Done in Traps

# Safe in traps:
# - Simple commands
# - Setting variables
# - echo/printf to stable file descriptors
# - Running cleanup functions

# Avoid in traps:
# - Complex commands that might fail
# - Commands that might block (I/O)
# - Commands that might send signals

# Better approach
cleanup() {
    # Just set a flag
    CLEANUP_REQUESTED=1
}

trap cleanup TERM INT

# Check flag in main loop
while true; do
    if [[ $CLEANUP_REQUESTED ]]; then
        break
    fi
    sleep 1
done

---

Summary

In this chapter, you learned:

• ✅ What signals are and how they work
• ✅ Common POSIX signals and their numbers
• ✅ The trap command and its syntax
• ✅ Cleanup on exit
• ✅ Ignoring signals
• ✅ Graceful shutdown
• ✅ DEBUG and ERR traps
• ✅ Real-world DevOps examples
• ✅ Signal safety best practices

---

Next Steps

Continue to the next chapter to learn about Job Control in Bash.

---

Previous Chapter: Process Management Next Chapter: Job Control