Process_management

Chapter 16: Process Management

Understanding and Controlling Linux Processes

16.1 Process Fundamentals

What is a Process?

A process is an instance of a running program. Each process has its own memory space, system resources, and execution context. In Linux, everything is treated as a file, and processes are no exception - they’re managed through the /proc filesystem.

                 Linux Process Architecture
+------------------------------------------------------------------+
|                                                                   |
|                         Process Structure                           |
|    +-------------------------------------------------------------+|
|    |                    Process Control Block (PCB)              ||
|    |    +---------------------------------------------------+   ||
|    |    | Process ID (PID)                                  |   ||
|    |    | Parent PID (PPID)                                 |   ||
|    |    | User ID (UID) / Group ID (GID)                  |   ||
|    |    | Process State                                     |   ||
|    |    | Program Counter (PC)                             |   ||
|    |    | CPU Registers                                    |   ||
|    |    | Memory Management Info                           |   ||
|    |    | I/O Status Information                           |   ||
|    |    | Accounting Information                            |   ||
|    |    +---------------------------------------------------+   ||
|    +-------------------------------------------------------------+|
|                              |                                      |
|                              v                                      |
|    +-------------------------------------------------------------+|
|    |                      Virtual Memory                           ||
|    |    +---------------------------------------------------+   ||
|    |    | Text Segment (Code)                              |   ||
|    |    | Data Segment (Initialized data)                 |   ||
|    |    | BSS Segment (Uninitialized data)                 |   ||
|    |    | Heap (Dynamic memory)                           |   ||
|    |    | Stack (Function calls, local variables)         |   ||
|    |    +---------------------------------------------------+   ||
|    +-------------------------------------------------------------+|
|                                                                   |
|                         Kernel Space                                |
|    +-------------------------------------------------------------+|
|    |  Process Table | File Descriptors | Network Sockets        ||
|    +-------------------------------------------------------------+|
|                                                                   |
+------------------------------------------------------------------+

Process States

                 Process State Diagram
+------------------------------------------------------------------+
|                                                                   |
|    States in /proc/[pid]/stat:                                     |
|                                                                   |
|    +---------+                                                    |
|    |    R   |  Running/Runnable - Executing or ready to execute|
|    +----+----+                                                    |
|         |                                                          |
|         | (Scheduler)                                             |
|         v                                                          |
|    +---------+      (I/O Wait)      +---------+                   |
|    |    S   |<---------------------->|    D   |                   |
|    | Sleeping|                       | Disk Sleep|                  |
|    +---------+                       +---------+                   |
|         |                                                          |
|         | (Stop signal)                                            |
|         v                                                          |
|    +---------+                                                    |
|    |    T   |  Stopped - Suspended (Ctrl+Z, SIGSTOP)            |
|    +---------+                                                    |
|         |                                                          |
|         | (Exit)                                                   |
|         v                                                          |
|    +---------+                                                    |
|    |    Z   |  Zombie - Terminated but not reaped by parent     |
|    +---------+                                                    |
|                                                                   |
|    Additional States:                                              |
|    +----------------------------------------------------------+  |
|    | I | Idle kernel thread                                   |  |
|    | K | Killable (sleeping, can be killed)                  |  |
|    | P | Parked (parked process)                             |  |
|    | X | Dead (should never be seen)                         |  |
|    +----------------------------------------------------------+  |
|                                                                   |
+------------------------------------------------------------------+

16.2 Viewing Processes

ps Command

# Basic process listing
ps                    # Current shell processes
ps -e                # All processes
ps -ef               # Full format
ps -eF               # Extra full
ps -eo pid,ppid,user,comm,%cpu,%mem,etime

# Common options
ps aux               # BSD style (a=all, u=user, x=extended)
ps -ef               # Standard style
ps -eLf              # Threads (LWP, NLWP)
ps -C nginx          # By command name
ps -p 1234           # By PID
ps -U username       # By user

# Sorting
ps -eo pid,ppid,%cpu,%mem,comm --sort=-%cpu    # By CPU
ps -eo pid,ppid,%cpu,%mem,comm --sort=-%mem   # By memory

# Tree view
ps -ef --forest
ps -ejH
pstree

# Custom output
ps -eo pid,ppid,state,cmd
ps -eo pid,ppid,state,etime,cmd

# Header (show column names)
ps -eo pid,ppid,user,comm,%cpu,%mem,etime --headers

top and htop

# top - Interactive process viewer
top                  # Default
top -u username      # User's processes only
top -p 1234         # Specific PID
top -d 1            # Refresh every 1 second
top -n 5            # Run 5 iterations
top -H              # Show threads
top -c              # Show command line
top -1              # Show per-CPU

# Interactive commands in top:
# P - sort by CPU
# M - sort by memory
# T - sort by time
# k - kill process
# r - renice process
# f - field manager
# W - write config

# htop - Enhanced top
htop                 # Interactive
htop -u username     # Filter by user
htop -p 1234,5678    # PIDs
htop -d 10          # Delay

# htop interactive:
# F1 - Help
# F2 - Setup columns
# F3 - Search
# F4 - Filter
# F5 - Tree view
# F6 - Sort by
# F7 - Nice -
# F8 - Nice +
# F9 - Kill
# F10 - Quit

Process Information in /proc

# Process information
ls -la /proc/1234/          # Process info
cat /proc/1234/status       # Detailed status
cat /proc/1234/cmdline      # Command line (null-separated)
cat /proc/1234/environ      # Environment (null-separated)
cat /proc/1234/fd/          # File descriptors
cat /proc/1234/maps         # Memory maps
cat /proc/1234/statm        # Memory statistics
cat /proc/1234/io           # I/O statistics

# Process status info
cat /proc/1234/status | grep -E "Name|Pid|State|Threads|VmSize|VmRSS"

# Maps (loaded libraries)
cat /proc/1234/maps

# File descriptors
ls -la /proc/1234/fd/
lsof -p 1234

16.3 Process Control

Sending Signals

# Signal types
# SIGTERM (15) - Graceful termination (default)
# SIGKILL (9) - Immediate termination
# SIGINT (2) - Interrupt (Ctrl+C)
# SIGSTOP (19) - Stop process
# SIGCONT (18) - Continue stopped process
# SIGHUP (1) - Hangup (reload config)
# SIGUSR1 (10) - User-defined
# SIGUSR2 (12) - User-defined

# Kill process
kill 1234              # Send SIGTERM
kill -9 1234          # Send SIGKILL
kill -KILL 1234       # By name

# By pattern
pkill nginx           # By name
pkill -9 nginx        # Force kill
pkill -u username    # User's processes
pkill -f "pattern"   # By command line

# Kill all processes
killall nginx         # All nginx
killall -9 nginx     # Force
killall -u username  # User's all

# Send specific signal
kill -HUP 1234       # Reload config
kill -USR1 1234     # User-defined

# Process groups
kill -TERM -1234     # Kill process group

Process Priority

# Nice value (-20 to 19, lower = higher priority)
# Root can go negative, regular users 0-19

# Start with priority
nice -n 10 command       # Lower priority (10)
nice -n -10 command       # Higher priority (root only)
nice --10 command         # Alternative syntax

# Change priority of running process
renice 10 1234           # Set priority
renice +5 -p 1234       # Increase (lower priority)
renice -5 -p 1234       # Decrease (higher priority, root)
renice 0 -u username    # All user's processes

# View priority
ps -eo pid,ppid,ni,cmd

# Tools
top                     # Press r to renice
htop                    # F7/F8 to nice

Background and Foreground

# Run in background
command &               # Run in background
nohup command &        # Immune to hangup
nohup command > output.txt 2>&1 &

# Jobs control
jobs                   # List jobs
jobs -l               # With PIDs

# Foreground/Background
fg                      # Bring to foreground
fg %1                  # Job 1 to foreground
fg %2                  # Job 2 to foreground

# Background
bg                     # Resume stopped job in background
bg %1                  # Resume job 1

# Disown from shell
disown %1              # Remove job from shell
disown -h %1          # Don't send SIGHUP
disown -a             # All jobs

# Screen/tmux
screen                 # New screen session
screen -S name         # Named session
screen -r              # Reattach
screen -ls            # List sessions

tmux new -s name      # New tmux
tmux attach -t name   # Attach
tmux ls               # List

16.4 Process Monitoring

Monitoring Tools

# top/htop (interactive)
top
htop
btop                   # Modern, GPU-accelerated

# Process specific info
ps aux | grep nginx
pidof nginx            # Get PID
pidof -s nginx        # Single PID
pidof -x nginx        # Include script names

# lsof - open files
lsof                   # All open files
lsof -p 1234          # Process files
lsof -u username      # User's files
lsof -i               # Network files
lsof -i :80           # Port 80
lsof -i TCP:80        # TCP port 80
lsof +D /var/log     # Files in directory

# ltrace/strace - system calls
strace -p 1234        # Trace process
strace -f -e openat   # Specific calls
strace -c            # Count syscalls
strace -o file.log   # To file

# Time and resource usage
time command
time -v command       # Verbose
/usr/bin/time -v command

# Continuous monitoring
watch -n 1 'ps aux | grep nginx'

Resource Usage

# CPU usage per process
ps -eo pid,comm,%cpu --sort=-%cpu | head -10
top                    # Press 1 for per-CPU

# Memory usage
ps -eo pid,comm,%mem --sort=-%mem | head -10
pmap -x 1234          # Memory map
cat /proc/1234/status | grep Vm

# I/O statistics
iotop                 # I/O usage
iotop -o             # Only active
iostat -x 1          # Disk I/O
pidstat -d 1         # Per-process I/O

# Network statistics
ss -tp               # TCP sockets
ss -up               # UDP sockets
netstat -tp          # Connections
nethogs              # Per-process network

# Thread count
ps -o nlwp= -p 1234
top -H -p 1234

16.5 Process Accounting

Process Accounting Setup

# Enable accounting
sudo apt install acct          # Debian/Ubuntu
sudo yum install psacct        # RHEL/CentOS

# Start service
sudo systemctl enable psacct
sudo systemctl start psacct

# Commands
lastcomm                  # Previous commands
lastcomm --user username  # User's commands
lastcomm --tty tty1       # Terminal
sa                         # Summary
sa -m                     # Per-user
sa -u                     # Per-command

# Process accounting file
ls -la /var/log/account/
cat /var/log/account/pacct*

Audit Logging

# Using auditd
sudo apt install auditd
sudo auditctl -w /usr/bin/nginx -p x -k nginx_exec
sudo ausearch -k nginx_exec

# Monitor process creation
sudo auditctl -a task,always
sudo ausearch -sc execve

16.6 Zombie and Orphan Processes

Understanding Zombie Processes

                 Zombie Process Lifecycle
+------------------------------------------------------------------+
|                                                                   |
|    Normal Process:                                                |
|                                                                   |
|    1. Parent forks child                                           |
|    2. Child runs and exits                                        |
|    3. Parent reads exit status (wait())                          |
|    4. Kernel removes process                                      |
|                                                                   |
|    Zombie Process:                                                |
|                                                                   |
|    1. Parent forks child                                           |
|    2. Child runs and exits                                        |
|    3. Parent doesn't call wait()                                  |
|    4. Child becomes ZOMBIE (dead but not cleaned)                |
|    5. Parent calls wait() → process removed                       |
|                                                                   |
|    Orphan Process:                                                 |
|                                                                   |
|    1. Parent forks child                                           |
|    2. Parent exits before child                                   |
|    3. Child adopted by init (PID 1)                               |
|    4. init calls wait() when child exits                          |
|                                                                   |
+------------------------------------------------------------------+

Finding and Handling

# Find zombie processes
ps aux | grep Z
ps -eo pid,ppid,state,cmd | grep Z
top                 # Z in status column

# Parent of zombie
ps -eo pid,ppid,state,cmd | grep Z

# Kill zombie
# Usually can't kill zombie - it's already dead
# Must kill parent process
kill -9 <parent_pid>
kill -SIGCHLD <parent_pid>

# If parent won't die
# Only option: restart parent or reboot

# Prevent zombies:
# - Properly call wait() in code
# - Use signal handlers for SIGCHLD
# - For orphaned: init handles automatically

16.7 Advanced Process Management

cgroups and Namespaces

# Using cgroups v2
# Create cgroup
sudo mkdir -p /sys/fs/cgroup/mygroup/
echo 500000000 > /sys/fs/cgroup/mygroup/memory.max
echo 100000 > /sys/fs/cgroup/mygroup/cpu.max

# Add process to cgroup
echo <pid> > /sys/fs/cgroup/mygroup/cgroup.procs

# Using systemd (easier)
systemd-run --scope -p MemoryMax=512M /bin/bash

# Resource control in systemd
# See Chapter 17 systemd

# Namespaces (unshare)
# Create new network namespace
sudo unshare -n bash
# Create new PID namespace
sudo unshare -p bash
# Create new mount namespace
sudo unshare -m bash

Performance Tuning

# Kernel tuning for process handling
# Process limits
# /etc/security/limits.conf
* soft nofile 65535
* hard nofile 65535
* soft nproc 65535
* hard nproc 65535

# Process scheduling
# nice values affected by:
# /etc/nicevalues.conf

# CPU affinity
taskset -p 0x1 1234      # CPU 0 only
taskset -cp 0,1 1234     # CPUs 0 and 1
taskset -a -p 0x3 1234   # All CPUs 0-1

16.8 Container Process Management

Containers and Processes

# Docker process listing
docker ps                 # Running containers
docker ps -a             # All containers
docker top container     # Processes in container

# crictl (Kubernetes)
crictl ps                # Container processes

# Namespaces inside container
# Each container has own:
# - PID namespace (process IDs)
# - Network namespace (network stack)
# - Mount namespace (filesystems)
# - UTS namespace (hostname)
# - IPC namespace (shared memory)
# - User namespace (UIDs/GIDs)

# Process limits in containers
docker run --cpus=2 container
docker run --memory=512m container
docker run --pids-limit=100 container

16.9 Interview Questions

Basic Questions

What is a process in Linux?
- An instance of a running program with its own memory space, PID, and resources
What are the different process states?
- R (Running), S (Sleeping), D (Disk Sleep), T (Stopped), Z (Zombie)
How do you list all processes?
- ps -ef, ps aux, top, htop
What is the difference between kill and pkill?
- kill takes PID, pkill can match by name, user, etc.
What is a zombie process?
- A terminated process whose parent hasn’t read its exit status

Intermediate Questions

What is the difference between nice and renice?
- nice sets priority at start, renice changes running process
How do you run a process in the background?
- Using &, nohup, screen, tmux, disown
What is the init process?
- PID 1, ancestor of all processes, handles orphaned processes
How do you find processes by name?
- pgrep, pkill, ps aux | grep
What is the difference between SIGTERM and SIGKILL?
- SIGTERM allows graceful shutdown, SIGKILL is immediate

Advanced Questions

How does the Linux scheduler work?
- Uses CFS (Completely Fair Scheduler), weights by nice value and runtime
What are cgroups and how are they used?
- Control groups for resource limiting (CPU, memory, I/O)
What are namespaces?
- Kernel feature for isolation (PID, network, mount, etc.)
How do you prevent zombie processes?
- Parent must call wait() or use SIGCHLD handler
What is the difference between OOM killer and process priority?
- OOM killer kills processes when memory exhausted, priority affects CPU scheduling

Summary

Process management is fundamental to Linux administration:

                 Quick Reference
+------------------------------------------------------------------+
|                                                                   |
|    Process Viewing:                                               |
|    +----------------------------------------------------------+  |
|    | ps aux                           | All processes          |  |
|    | top/htop                        | Interactive viewer     |  |
|    | pgrep pattern                   | Find by pattern       |  |
|    | lsof -p PID                     | Open files            |  |
|    +----------------------------------------------------------+  |
|                                                                   |
|    Process Control:                                               |
|    +----------------------------------------------------------+  |
|    | kill PID                         | Terminate gracefully  |  |
|    | kill -9 PID                      | Force kill            |  |
|    | pkill name                       | Kill by name          |  |
|    | renice N PID                     | Change priority       |  |
|    | nice -n N command                | Start with priority   |  |
|    +----------------------------------------------------------+  |
|                                                                   |
|    Background/Foreground:                                          |
|    +----------------------------------------------------------+  |
|    | command &                        | Run in background     |  |
|    | Ctrl+Z, bg                       | Background stopped    |  |
|    | fg                              | Bring to foreground   |  |
|    | nohup command &                  | Immune to hangup      |  |
|    | screen/tmux                     | Terminal multiplexer  |  |
|    +----------------------------------------------------------+  |
|                                                                   |
|    Key Terms:                                                     |
|    +----------------------------------------------------------+  |
|    | PID       | Process ID                                    |  |
|    | PPID     | Parent PID                                   |  |
|    | UID/GID  | User/Group ID                                |  |
|    | Nice     | Scheduling priority (-20 to 19)             |  |
|    | Zombie   | Dead but not cleaned                         |  |
|    +----------------------------------------------------------+  |
|                                                                   |
+------------------------------------------------------------------+