Coprocesses

Chapter 41: Coprocesses in Bash

Overview

Coprocesses are a powerful feature in Bash that allow you to create background processes with two-way communication. They enable you to send input to and receive output from background processes, making them ideal for interactive scripts and scenarios requiring continuous communication between the shell and a process.

---

What is a Coprocess?

A coprocess is a background process that has its stdin and stdout connected to the parent shell through named pipes. Unlike regular background jobs, coprocesses allow bidirectional communication.

┌────────────────────────────────────────────────────────────────┐
│                    Coprocess Architecture                            │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│   Shell                                                       │
│   ─────                                                       │
│                                                                │
│   ┌─────────────────────┐                                     │
│   │   Coprocess        │                                     │
│   │   (bg process)     │──── stdout ───►                     │
│   │                    │◄─── stdin ────                     │
│   └─────────────────────┘                                     │
│                                                                │
│   write -p "data"   ───►   coprocess input                  │
│   read -p variable  ◄───    coprocess output                 │
│                                                                │
│   Key difference from regular background jobs:                   │
│   - Regular bg job: stdout only, one-way                      │
│   - Coprocess: stdin AND stdout, two-way                     │
│                                                                │
└────────────────────────────────────────────────────────────────┘

---

Basic Syntax

Starting a Coprocess

# Basic coprocess syntax
coproc NAME { command; }

# Example with bc calculator
coproc BC { bc -l; }

# Using named pipes
coproc myprocess { while read line; do echo "Processed: $line"; done; }

Accessing File Descriptors

When you start a coprocess, Bash creates these variables:

# ${NAME[0]} - PID of the coprocess
# ${NAME[1]} - File descriptor for writing to coprocess stdin
# ${NAME[2]} - File descriptor for reading from coprocess stdout

# Example:
coproc BC { bc -l; }

# BC_PID (or ${BC[0]}) contains the PID
echo "BC PID: $BC_PID"

# Write to stdin using ${BC[1]}
echo "scale=2" >&${BC[1]}

# Read from stdout using ${BC[0]}
read -r result <&${BC[0]}
echo "Result: $result"

---

Practical Examples

Interactive Calculator

#!/usr/bin/env bash
# Calculator using coprocess - DevOps use case for calculations

set -euo pipefail

# Start bc as coprocess for precision calculations
coproc BC { bc -l; }

# Set precision
echo "scale=4" >&${BC[1]}

# Perform various calculations
echo "10 + 5" >&${BC[1]}
read -r sum <&${BC[0]}
echo "10 + 5 = $sum"

echo "100 / 3" >&${BC[1]}
read -r result <&${BC[0]}
echo "100 / 3 = $result"

echo "2^10" >&${BC[1]}
read -r power <&${BC[0]}
echo "2^10 = $power"

# Clean quit
echo "quit" >&${BC[1]}

# Kill the coprocess
kill ${BC_PID} 2>/dev/null || true
echo "Calculator closed"

Continuous Data Processing

#!/usr/bin/env bash
# Process log entries through awk coprocess in real-time

set -euo pipefail

# Start awk coprocess for log processing
coproc AWK {
    awk '{
        # Count by log level
        if (/ERROR/) errors++
        if (/WARN/) warns++
        if (/INFO/) infos++
        END {
            print "ERRORS:", errors
            print "WARNINGS:", warns
            print "INFO:", infos
        }
    }';
}

# Send log data to process
cat /var/log/syslog | while read -r line; do
    echo "$line" >&${AWK[1]}
done

# Close input to trigger END block
exec {AWK[1]}>&-

# Read final results
while read -r line <&${AWK[0]}; do
    echo "Results: $line"
done

Database Connection Pool

#!/usr/bin/env bash
# Maintain persistent database connection for multiple queries

set -euo pipefail

# Start PostgreSQL interactive session as coprocess
coproc PSQL {
    psql -h localhost -U appuser -d myapp
}

# Wait for connection to establish
sleep 2

# Execute multiple queries through same connection
echo "SELECT version();" >&${PSQL[1]}
read -r version <&${PSQL[0]}
echo "$version"

echo "SELECT current_database();" >&${PSQL[1]}
read -r dbname <&${PSQL[0]}
echo "$dbname"

echo "SELECT count(*) FROM users;" >&${PSQL[1]}
read -r count <&${PSQL[0]}
echo "$count"

# Quit gracefully
echo "\q" >&${PSQL[1]}

---

Advanced Usage

Multiple Coprocesses

#!/usr/bin/env bash
# Use multiple coprocesses simultaneously

set -euo pipefail

# Start two coprocesses for parallel processing
coproc PROCESS1 { sed 's/.*/P1: &/'; }
coproc PROCESS2 { sed 's/.*/P2: &/'; }

# Send data to first process
echo "data1" >&${PROCESS1[1]}
echo "data2" >&${PROCESS2[1]}

# Read results from both
read -r out1 <&${PROCESS1[0]}
read -r out2 <&${PROCESS2[0]}

echo "Process 1: $out1"
echo "Process 2: $out2"

# Cleanup
kill ${PROCESS1_PID} ${PROCESS2_PID} 2>/dev/null || true

Real-time Server Communication

#!/usr/bin/env bash
# Communicate with network service in real-time

set -euo pipefail

# Connect to server (using nc for demonstration)
coproc SOCAT { nc localhost 8080; }

# Send HTTP request
echo -e "GET /health HTTP/1.1\r\nHost: localhost\r\n\r" >&${SOCAT[1]}

# Read response
while read -r line <&${SOCAT[0]}; do
    echo "$line"
done

# Close connection
exec {SOCAT[1]}>&-

---

Error Handling

Proper Cleanup

#!/usr/bin/env bash
# Coprocess with proper error handling and cleanup

set -euo pipefail

cleanup() {
    # Kill coprocess if running
    if [[ -n "${BC_PID:-}" ]] && kill -0 "$BC_PID" 2>/dev/null; then
        echo "quit" >&${BC[1]} 2>/dev/null || true
        kill "$BC_PID" 2>/dev/null || true
    fi
}

trap cleanup EXIT

# Start coprocess
coproc BC { bc -l; }

# Use coprocess
echo "scale=2; 22/7" >&${BC[1]}
read -r result <&${BC[0]}
echo "Result: $result"

Timeout Handling

#!/usr/bin/env bash
# Coprocess with timeout

set -euo pipefail

coproc BC { bc -l; }

# Set timeout
timeout=5

echo "100/3" >&${BC[1]}

# Read with timeout
if read -t "$timeout" -r result <&${BC[0]}; then
    echo "Result: $result"
else
    echo "Timeout waiting for response"
    kill ${BC_PID} 2>/dev/null || true
fi

---

DevOps Use Cases

CI/CD Pipeline Integration

#!/usr/bin/env bash
# Use coprocess for continuous integration build process

set -euo pipefail

# Start build process
coproc BUILD {
    make build 2>&1
}

# Monitor build progress
while read -r line <&${BUILD[0]}; do
    echo "[BUILD] $line"

    # Check for errors
    if echo "$line" | grep -q "ERROR"; then
        echo "Build failed!"
        kill ${BUILD_PID} 2>/dev/null || true
        exit 1
    fi

    # Check for success
    if echo "$line" | grep -q "Build complete"; then
        echo "Build successful!"
    fi
done

Metrics Collection

#!/usr/bin/env bash
# Collect system metrics continuously

set -euo pipefail

# Start metrics collector
coproc METRICS {
    while true; do
        echo "cpu:$(top -bn1 | grep "Cpu(s)" | awk '{print $2}')"
        echo "mem:$(free | grep Mem | awk '{print ($3/$2) * 100}')"
        sleep 5
    done
}

# Collect for 30 seconds
end=$((SECONDS + 30))
while [[ $SECONDS -lt $end ]]; do
    if read -t 2 -r metric <&${METRICS[0]}; then
        echo "[METRIC] $metric"
    fi
done

kill ${METRICS_PID} 2>/dev/null || true

---

Summary

In this chapter, you learned:

• ✅ What coprocesses are and how they work
• ✅ Basic syntax and file descriptor access
• ✅ Two-way communication with coprocesses
• ✅ Practical examples for DevOps use cases
• ✅ Multiple coprocess handling
• ✅ Error handling and cleanup
• ✅ Real-world DevOps applications

---

Next Steps

Continue to the next chapter to learn about Process Substitution.

---

Previous Chapter: Container Automation Next Chapter: Process Substitution