Job_control

Chapter 25: Job Control in Bash

Overview

Job control allows you to manage multiple processes (jobs) running in a shell session. This chapter covers how to run jobs in the background, bring them to the foreground, and manage them effectively - essential skills for DevOps engineers.

---

Understanding Jobs

What is a Job?

A job is a process or group of processes started by the shell. Each job has:

• Job ID - Shell-relative identifier (%1, %2, etc.)
• Process Group ID - System-level identifier
• Status - Running, Stopped, Done

┌────────────────────────────────────────────────────────────────┐
│                    Job Control Flow                             │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│   ┌─────────┐                                                  │
│   │ Shell   │                                                  │
│   └────┬────┘                                                  │
│        │                                                        │
│    ┌───┴───┐                                                    │
│    │ Job 1 │ (Foreground)                                      │
│    └───┬───┘                                                    │
│        │                                                        │
│    ┌───┴───┐                                                    │
│    │ Job 2 │ (Background)                                      │
│    └───────┘                                                    │
│                                                                │
│   Job Control:                                                  │
│   - Start jobs in background (&)                              │
│   - Stop jobs (Ctrl+Z)                                        │
│   - Bring to foreground (fg)                                  │
│   - send to background (bg)                                   │
│                                                                │
└────────────────────────────────────────────────────────────────┘

---

Job Commands

jobs - List Jobs

# List all jobs
jobs

# List with process IDs
jobs -l

# List only running jobs
jobs -r

# List only stopped jobs
jobs -s

fg - Bring to Foreground

# Bring job 1 to foreground
fg %1

# Bring current job (or +)
fg %+

# Bring previous job (or -)
fg %-

# By process group
fg %%   # Current
fg %+   # Current
fg %-   # Previous

bg - Send to Background

# Resume job 1 in background
bg %1

# Resume current stopped job
bg

# Resume job in background
bg %-

---

Starting Background Jobs

Basic Background Execution

# Start in background
./long_task.sh &

# Start multiple background jobs
./task1.sh &
./task2.sh &
./task3.sh &

# Start with output redirection
./script.sh > output.log 2>&1 &

# Using nohup (survives terminal close)
nohup ./script.sh &

# Using setsid (new session)
setsid ./script.sh &

---

Job States

Running

# Start running job
./script.sh &

# Verify running
jobs
# [1]+  Running                 ./script.sh &

Stopped (Suspended)

# Stop foreground job
# Press Ctrl+Z

# Verify stopped
jobs
# [1]+  Stopped                 ./script.sh

Done

# Check when job completes
jobs
# [1]+  Done                    ./script.sh

# Or history shows completed
jobs -l
# [1]+  1234 Done              ./script.sh

---

Managing Multiple Jobs

Parallel Execution

#!/usr/bin/env bash
# Run tasks in parallel

# Start all tasks
./task1.sh &
PID1=$!
./task2.sh &
PID2=$!
./task3.sh &
PID3=$!

# Wait for all
wait $PID1 $PID2 $PID3

echo "All tasks complete"

Controlled Parallelism

#!/usr/bin/env bash
# Run with limited parallelism

MAX_JOBS=4
COUNTER=0

for item in {1..20}; do
    # Wait if max jobs reached
    while (( $(jobs -p | wc -l) >= MAX_JOBS )); do
        sleep 0.5
    done

    # Run task in background
    ./process_item.sh $item &
    ((COUNTER++))
done

# Wait for remaining jobs
wait

echo "Processed $COUNTER items"

Wait with Timeout

#!/usr/bin/env bash

# Start background job
./long_task.sh &
PID=$!

# Wait with timeout
TIMEOUT=60
count=0

while kill -0 $PID 2>/dev/null; do
    if [[ $count -ge $TIMEOUT ]]; then
        echo "Timeout, killing process..."
        kill $PID
        break
    fi
    sleep 1
    ((count++))
done

echo "Job completed or timed out"

---

Process Group Management

Session and Process Group

# Get session ID
ps -o sid= -p $$

# Get process group
ps -o pgrp= -p $$

# Move process to new group
setsid ./script.sh

# Kill entire process group
kill -TERM -$(pgrep -f process_name)

---

Real-World DevOps Examples

Parallel Log Processing

#!/usr/bin/env bash
# Process multiple log files in parallel

LOG_FILES=(
    "/var/log/nginx/access.log"
    "/var/log/nginx/error.log"
    "/var/log/app/application.log"
    "/var/log/syslog"
)

process_log() {
    local logfile="$1"
    echo "Processing: $logfile"

    # Extract metrics
    local lines=$(wc -l < "$logfile")
    local errors=$(grep -c ERROR "$logfile" 2>/dev/null || echo 0)

    echo "$logfile: $lines lines, $errors errors"
}

# Process in parallel
for log in "${LOG_FILES[@]}"; do
    process_log "$log" &
done

wait
echo "All logs processed"

Concurrent API Requests

#!/usr/bin/env bash
# Make concurrent API requests

URLS=(
    "https://api1.example.com/data"
    "https://api2.example.com/data"
    "https://api3.example.com/data"
    "https://api4.example.com/data"
)

fetch_data() {
    local url="$1"
    curl -s "$url" | jq -c '.'
}

for url in "${URLS[@]}"; do
    fetch_data "$url" &
done

wait
echo "All requests complete"

Background Service Monitor

#!/usr/bin/env bash
# Monitor multiple services

SERVICES=("nginx" "mysql" "redis" "docker")

monitor_service() {
    local service="$1"

    while true; do
        if systemctl is-active --quiet "$service"; then
            : # Service running
        else
            echo "$(date): $service is DOWN!"
        fi
        sleep 10
    done
}

# Start monitors in background
for service in "${SERVICES[@]}"; do
    monitor_service "$service" &
done

echo "Monitors started"

# Wait (or use trap to cleanup)
wait

---

Job Control in Scripts

Disown Jobs

#!/usr/bin/env bash

# Start job and disown
./background_task.sh &
disown

# Disown specific job
./task.sh &
disown %1

# Disown all jobs
disown -a

# Keep job in job table but ignore SIGHUP
disown -h %1

Protecting Jobs from HUP

#!/usr/bin/env bash

# nohup - ignore hangup signals
nohup ./script.sh &

# nohup with output redirect
nohup ./script.sh > output.log 2>&1 &

# Using setsid (new session)
setsid ./script.sh &

# Using screen/tmux (see next section)

---

Using screen and tmux

GNU Screen

# Start named screen session
screen -S mysession

# Detach from screen
Ctrl+A D

# List screens
screen -ls

# Reattach
screen -r mysession

# Reattach or create
screen -d -RR mysession

# Run command in screen
screen -dmS mysession ./script.sh

# Send command to screen
screen -S mysession -X stuff './script.sh\n'

tmux (Modern Alternative)

# Start tmux session
tmux new -s mysession

# Detach from tmux
Ctrl+B D

# List sessions
tmux ls

# Reattach
tmux attach -t mysession

# Split window vertically
Ctrl+B "

# Split window horizontally
Ctrl+B %

# Run command in tmux
tmux new -d -s mysession './script.sh'

# Send command to tmux session
tmux send-keys -t mysession './script.sh' C-m

---

wait Command

Waiting for Jobs

# Wait for specific job
./task.sh &
wait $!

# Wait for all background jobs
./task1.sh &
./task2.sh &
wait

# Wait with specific job
wait %1 %2

# Wait returns exit status of job
./script.sh &
wait $!
echo "Exit status: $?"

---

Troubleshooting Job Control

Common Issues

# Job not starting in background?
# Check for background jobs limit
ulimit -u

# Too many background jobs?
jobs -l
# Kill excess jobs
kill $(jobs -p)

# Jobs not responding to Ctrl+C?
# Check for disowned jobs
ps aux | grep script

# Kill manually
pkill -f script_name

---

Summary

In this chapter, you learned:

• ✅ Job basics and job control concepts
• ✅ jobs, fg, bg commands
• ✅ Starting background jobs
• ✅ Job states (running, stopped, done)
• ✅ Parallel execution patterns
• ✅ Controlled parallelism
• ✅ Process groups
• ✅ Real-world DevOps examples
• ✅ nohup, disown, setsid
• ✅ screen and tmux for persistent sessions
• ✅ wait command and job exit status
• ✅ Troubleshooting job control

---

Next Steps

Continue to the next chapter to learn about Environment Variables.

---

Previous Chapter: Signals and Traps Next Chapter: Environment Variables