Docker

Chapter 58: Docker - Complete Guide

Mastering Container Technology for DevOps

58.1 Understanding Docker Architecture

What is Docker?

Docker is an open platform for developing, shipping, and running applications using containerization technology. Containers are lightweight, standalone packages that include everything needed to run software: code, runtime, libraries, and settings.

                     Docker vs Traditional Virtualization
+------------------------------------------------------------------+
|                                                                   |
|    Traditional Virtual Machines                                      |
|    +----------------------------------------------------------+   |
|    |  +----------+  +----------+  +----------+                 |   |
|    |  | Guest OS |  | Guest OS |  | Guest OS |                 |   |
|    |  +----------+  +----------+  +----------+                 |   |
|    |       |             |             |                        |   |
|    |       v             v             v                        |   |
|    |  +-----------------------------------------------+          |   |
|    |  |            Hypervisor (VMware, Hyper-V)         |          |   |
|    |  +-----------------------------------------------+          |   |
|    |                          |                              |   |
|    |                          v                              |   |
|    |  +-----------------------------------------------+          |   |
|    |  |              Hardware (CPU, RAM, Disk)          |          |   |
|    |  +-----------------------------------------------+          |   |
|    |                                                           |   |
|    |  Drawbacks:                                               |   |
|    |  - Heavy (full OS per VM)                               |   |
|    |  - Slow to start                                         |   |
|    |  - Resource intensive                                    |   |
|    +----------------------------------------------------------+   |
|                                                                   |
|    Docker Containers                                               |
|    +----------------------------------------------------------+   |
|    |  +----------+  +----------+  +----------+                 |   |
|    |  |Container |  |Container |  |Container |                 |   |
|    |  +----------+  +----------+  +----------+                 |   |
|    |       |             |             |                        |   |
|    |       v             v             v                        |   |
|    |  +-----------------------------------------------+          |   |
|    |  |         Docker Engine / Containerd              |          |   |
|    |  +-----------------------------------------------+          |   |
|    |                          |                              |   |
|    |                          v                              |   |
|    |  +-----------------------------------------------+          |   |
|    |  |              Host OS (Linux Kernel)              |          |   |
|    |  +-----------------------------------------------+          |   |
|    |                          |                              |   |
|    |                          v                              |   |
|    |  +-----------------------------------------------+          |   |
|    |  |              Hardware (CPU, RAM, Disk)          |          |   |
|    |  +-----------------------------------------------+          |   |
|    |                                                           |   |
|    |  Benefits:                                               |   |
|    |  - Lightweight (share OS kernel)                        |   |
|    |  - Fast to start (seconds)                              |   |
|    |  - Resource efficient                                    |   |
|    |  - Portable                                               |   |
|    +----------------------------------------------------------+   |
|                                                                   |
+------------------------------------------------------------------+

Docker Architecture Components

                     Docker Architecture Deep Dive
+------------------------------------------------------------------+
|                                                                   |
|                        Docker Client                                  |
|    +-------------------------------------------------------------+ |
|    |  - CLI (docker command)                                      | |
|    |  - REST API                                                  | |
|    |  - Communicates with Docker Daemon                          | |
|    +-------------------------------------------------------------+ |
|                              |                                      |
|                              v                                      |
|                        Docker Daemon                                 |
|    +-------------------------------------------------------------+ |
|    |  - dockerd process                                           | |
|    |  - Image management                                          | |
|    |  - Container lifecycle                                       | |
|    |  - Network management                                        | |
|    |  - Volume management                                         | |
|    +-------------------------------------------------------------+ |
|                              |                                      |
|              +---------------+---------------+                      |
|              |               |               |                      |
|              v               v               v                      |
|    +------------+   +------------+   +------------+              |
|    | Containerd |   |   runc    |   |  Storage   |              |
|    | (Container|   | (Runtime)  |   | (GraphDB)  |              |
|    |  Runtime)  |   |            |   |            |              |
|    +------------+   +------------+   +------------+              |
|                                                                   |
|                        Registry                                     |
|    +-------------------------------------------------------------+ |
|    |  - Docker Hub                                                 | |
|    |  - Private registries                                        | |
|    |  - Stores images                                             | |
|    +-------------------------------------------------------------+ |
|                                                                   |
+------------------------------------------------------------------+

58.2 Docker Installation and Configuration

Installing Docker

# =============================================================================
# ARCH LINUX
# =============================================================================

# Install Docker
sudo pacman -S docker

# Start and enable Docker
sudo systemctl enable --now docker

# Add user to docker group
sudo usermod -aG docker $USER

# Verify installation
docker --version
docker info

# Test Docker
docker run hello-world

# =============================================================================
# UBUNTU/DEBIAN
# =============================================================================

# Update packages
sudo apt update

# Install dependencies
sudo apt install -y apt-transport-https ca-certificates curl gnupg lsb-release

# Add Docker GPG key
curl -fsSL https://download.docker.com/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# Add Docker repository
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io

# =============================================================================
# RED HAT / CENTOS
# =============================================================================

# Install dependencies
sudo yum install -y yum-utils

# Add Docker repository
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

# Install Docker
sudo yum install -y docker-ce docker-ce-cli containerd.io

# Start Docker
sudo systemctl start docker
sudo systemctl enable docker

Docker Configuration

# =============================================================================
# DOCKER DAEMON CONFIGURATION
# =============================================================================

# Create daemon configuration
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<EOF
{
  "storage-driver": "overlay2",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  },
  "live-restore": true,
  "default-address-pools": [
    {
      "base": "172.17.0.0/16",
      "size": 24
    }
  ]
}
EOF

# Restart Docker
sudo systemctl restart docker

# =============================================================================
# POST-INSTALLATION SETUP
# =============================================================================

# Create docker network for custom subnet
docker network create \
  --driver=bridge \
  --subnet=172.20.0.0/16 \
  app-network

# Create persistent volume
docker volume create --name postgres-data

# List networks
docker network ls

# List volumes
docker volume ls

58.3 Working with Docker Images

Image Concepts

                     Docker Image Layers
+------------------------------------------------------------------+
|                                                                   |
|    Image = Collection of Read-Only Layers                          |
|                                                                   |
|    +----------------------------------------------------------+   |
|    |  Container Layer (writable)                               |   |
|    +----------------------------------------------------------+   |
|    |            |            |            |                     |   |
|    |  Layer 4   |  Layer 3  |  Layer 2  |  Layer 1         |   |
|    |  (CMD)     |  (RUN)    |  (COPY)   |  (FROM)          |   |
|    +----------------------------------------------------------+   |
|                                                                   |
|    Key Points:                                                    |
|    +----------------------------------------------------------+   |
|    |  - Each instruction creates a new layer                    |   |
|    |  - Layers are shared between images                         |   |
|    |  - Only changes are stored (space efficient)              |   |
|    |  - Copy-on-write: changes copy layer to container          |   |
|    +----------------------------------------------------------+   |
|                                                                   |
+------------------------------------------------------------------+

Image Commands

# =============================================================================
# PULLING IMAGES
# =============================================================================

# Pull latest image
docker pull nginx:latest

# Pull specific tag
docker pull nginx:1.25-alpine

# Pull all tags
docker pull -a nginx

# Pull with digest
docker pull nginx@sha256:abc123...

# Pull from private registry
docker pull registry.example.com:5000/myimage:v1

# =============================================================================
# LISTING IMAGES
# =============================================================================

# List all images
docker images
docker image ls

# List dangling images (none tag)
docker images -f dangling=true

# List images with filter
docker images --filter "reference=nginx:*"

# Format output
docker images --format "{{.Repository}}:{{.Tag}} {{.Size}}"

# =============================================================================
# INSPECTING IMAGES
# =============================================================================

# Show image details
docker image inspect nginx:latest

# Show layers
docker history nginx:latest

# Show full JSON
docker inspect nginx:latest --format='{{json .Config}}'

# =============================================================================
# REMOVING IMAGES
# =============================================================================

# Remove image
docker rmi nginx:latest

# Force remove
docker rmi -f nginx:latest

# Remove unused images
docker image prune

# Remove all unused images
docker image prune -a

# Remove dangling images
docker image prune -f

# =============================================================================
# TAGGING IMAGES
# =============================================================================

# Tag image for local registry
docker tag nginx:latest myregistry:5000/nginx:latest

# Tag for Docker Hub
docker tag nginx:latest username/nginx:latest

# =============================================================================
# PUSHING IMAGES
# =============================================================================

# Push to registry
docker push myregistry:5000/nginx:latest

# Push all tags
docker push myregistry:5000/nginx

58.4 Working with Containers

Container Lifecycle

                     Container Lifecycle
+------------------------------------------------------------------+
|                                                                   |
|    1. Create (docker create)                                      |
|    +----------------------------------------------------------+   |
|    |  - Creates container from image                          |   |
|    |  - Does not start container                              |   |
|    |  - Returns container ID                                   |   |
|    +----------------------------------------------------------+   |
|                         |                                          |
|                         v                                          |
|    2. Start (docker start)                                       |
|    +----------------------------------------------------------+   |
|    |  - Container begins running                               |   |
|    |  - Executes entrypoint/cmd                               |   |
|    |  - Container can be started/stopped multiple times      |   |
|    +----------------------------------------------------------+   |
|                         |                                          |
|                         v                                          |
|    3. Run (docker run) = create + start                         |
|    +----------------------------------------------------------+   |
|    |  - Creates and starts in one command                     |   |
|    |  - Most common way to run containers                     |   |
|    +----------------------------------------------------------+   |
|                         |                                          |
|                         v                                          |
|    4. Stop (docker stop)                                         |
|    +----------------------------------------------------------+   |
|    |  - Graceful shutdown (SIGTERM, then SIGKILL)             |   |
|    |  - Default 10 second timeout                              |   |
|    +----------------------------------------------------------+   |
|                         |                                          |
|                         v                                          |
|    5. Remove (docker rm)                                         |
|    +----------------------------------------------------------+   |
|    |  - Only stopped containers can be removed                 |   |
|    |  - Use -f to force remove running container             |   |
|    +----------------------------------------------------------+   |
|                                                                   |
+------------------------------------------------------------------+

Container Commands

# =============================================================================
# RUNNING CONTAINERS
# =============================================================================

# Basic run
docker run nginx

# Run in detached mode
docker run -d nginx

# Run with name
docker run -d --name my-nginx nginx

# Run with port mapping
docker run -d -p 8080:80 nginx
docker run -d -p 192.168.1.10:8080:80 nginx

# Run with volume
docker run -d -v /data:/var/www/html nginx

# Run with environment variables
docker run -d -e APP_ENV=production -e DB_HOST=localhost nginx

# Run with restart policy
docker run -d --restart=always nginx
docker run -d --restart=on-failure:5 nginx
docker run -d --restart=unless-stopped nginx

# Run with resource limits
docker run -d --memory=512m --cpus=0.5 nginx

# Run interactively
docker run -it ubuntu /bin/bash
docker run -it --rm ubuntu  # Auto-remove on exit

# =============================================================================
# MANAGING CONTAINERS
# =============================================================================

# List running containers
docker ps

# List all containers
docker ps -a

# List with format
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

# Start container
docker start my-nginx

# Stop container
docker stop my-nginx
docker stop -t 30 my-nginx  # 30 second timeout

# Restart container
docker restart my-nginx

# Pause container (freezes CPU)
docker pause my-nginx
docker unpause my-nginx

# Remove container
docker rm my-nginx
docker rm -f my-nginx  # Force remove
docker rm $(docker ps -aq)  # Remove all

# =============================================================================
# LOGS AND EXEC
# =============================================================================

# View logs
docker logs my-nginx
docker logs -f my-nginx  # Follow
docker logs --tail 100 my-nginx
docker logs --since 1h my-nginx

# Execute command in container
docker exec my-nginx ls -la
docker exec -it my-nginx /bin/bash

# Copy files
docker cp my-nginx:/etc/nginx/nginx.conf ./nginx.conf
docker cp ./index.html my-nginx:/usr/share/nginx/html/

# Inspect container
docker inspect my-nginx
docker inspect --format='{{.NetworkSettings.IPAddress}}' my-nginx

58.5 Docker Networking

Network Types

                     Docker Network Drivers
+------------------------------------------------------------------+
|                                                                   |
|    1. Bridge (default)                                           |
|    +----------------------------------------------------------+   |
|    |  - Default network for containers                         |   |
|    |  - Creates docker0 bridge (172.17.0.0/16)               |   |
|    |  - Containers can communicate with each other            |   |
|    |  - External access via port mapping                       |   |
|    +----------------------------------------------------------+   |
|                                                                   |
|    2. Host                                                       |
|    +----------------------------------------------------------+   |
|    |  - Container shares host's network namespace             |   |
|    |  - No network isolation                                   |   |
|    |  - Best performance                                       |   |
|    +----------------------------------------------------------+   |
|                                                                   |
|    3. Overlay                                                    |
|    +----------------------------------------------------------+   |
|    |  - Multi-host networking (Docker Swarm)                  |   |
|    |  - VXLAN encapsulation                                   |   |
|    |  - Service discovery                                      |   |
|    +----------------------------------------------------------+   |
|                                                                   |
|    4. Macvlan                                                   |
|    +----------------------------------------------------------+   |
|    |  - Container gets MAC address                             |   |
|    |  - Appears as physical device on network                |   |
|    |  - Direct network access                                  |   |
|    +----------------------------------------------------------+   |
|                                                                   |
|    5. None                                                       |
|    +----------------------------------------------------------+   |
|    |  - Disables all networking                                |   |
|    |  - Container is isolated                                  |   |
|    +----------------------------------------------------------+   |
|                                                                   |
+------------------------------------------------------------------+

Creating and Managing Networks

# List all networks
docker network ls
# Output:
# NETWORK ID     NAME      DRIVER    SCOPE
# abc123         bridge    bridge    local
# def456         host      host      local
# ghi789         none      null      local
# jkl012         mynet     bridge    local

# Create bridge network
docker network create -d bridge mybridge

# Create with custom subnet
docker network create \
  --subnet=192.168.100.0/24 \
  --gateway=192.168.100.1 \
  mynetwork

# Create with specific driver
docker network create -d overlay myoverlay

# Inspect network
docker network inspect bridge

# Connect container to network
docker network connect mynetwork container_name

# Disconnect container
docker network disconnect mynetwork container_name

# Remove network
docker network rm mynetwork

# Prune unused networks
docker network prune

DNS and Service Discovery

# Containers can resolve each other by name
# Docker provides embedded DNS at 127.0.0.11

# Create network with DNS
docker network create \
  --internal \
  --dns=8.8.8.8 \
  internal-net

# Alias for service discovery
docker network connect --alias api mynetwork container_name

# Test DNS resolution
docker run --rm --network mynetwork alpine nslookup container_name
docker run --rm --network mynetwork alpine ping -c 1 other_container

Port Mapping and Publishing

# Basic port mapping (host:container)
docker run -p 8080:80 nginx

# Listen on specific host IP
docker run -p 127.0.0.1:8080:80 nginx

# UDP port
docker run -p 8080:80/udp nginx

# Random host port
docker run -P nginx
# Docker assigns random port from 32768-60999

# Port range
docker run -p 8000-9000:80 nginx
# Maps 8000-9000 to container port 80

# Multiple ports
docker run -p 8080:80 -p 8443:443 nginx

# Specific port with protocol
docker run -p 53:53/tcp -p 53:53/udp dns-container

58.6 Storage Drivers and Volumes

Storage Driver Comparison

+------------------------------------------------------------------+
|                    DOCKER STORAGE DRIVERS                          |
+------------------------------------------------------------------+
|                                                                        |
|  +-------------+-------------+------------------------------------+    |
|  | Driver      | Use Case   | Notes                               |    |
|  +-------------+-------------+------------------------------------+    |
|  | overlay2    | Production | Default, best performance          |    |
|  | fuse3       | FUSE      | For rootless containers            |    |
|  | btrfs       | Btrfs FS  | Requires Btrfs filesystem          |    |
|  | zfs         | ZFS       | Requires ZFS filesystem            |    |
|  | vfs         | Testing   | Simple, no copy-on-write           |    |
|  | devicemapper| Legacy    | Deprecated in favor of overlay2    |    |
|  +-------------+-------------+------------------------------------+    |
|                                                                        |
|  For production: overlay2 (or fuse-overlayfs for rootless)            |
|                                                                        |
+------------------------------------------------------------------+

Docker Volumes

# Create volume
docker volume create myvolume

# List volumes
docker volume ls

# Inspect volume
docker volume inspect myvolume

# Mount volume to container
docker run -v myvolume:/data container

# Mount host directory
docker run -v /host/path:/container/path container

# Read-only mount
docker run -v myvolume:/data:ro container

# Mount with SELinux label
docker run -v /data:/data:z container
docker run -v /data:/data:Z container

# Create volume with specific driver
docker volume create \
  --driver local \
  --name mydata \
  -o type=none \
  -o o=bind \
  -o device=/path/to/data

# Remove unused volumes
docker volume prune

# Remove specific volume
docker volume rm myvolume

Bind Mounts

# Mount config files
docker run -v ./nginx.conf:/etc/nginx/nginx.conf:ro nginx

# Mount source code (development)
docker run -v $(pwd):/app -w /app node:alpine npm start

# Mount with ro/rw
docker run -v $(pwd):/app:ro node:alpine

# tmpfs mount (memory only)
docker run --tmpfs /tmp:size=100m,mode=1777 container

58.7 Resource Limits

Memory Limits

# Hard memory limit
docker run -m 512m nginx

# Memory limit with reservation
docker run -m 512m --memory-reservation=256m nginx

# Memory swap limit
docker run -m 512m --memory-swap=1g nginx

# Disable swap (set to same as memory)
docker run -m 512m --memory-swap=512m nginx

# Kernel memory limit
docker run -m 512m --kernel-memory=256m nginx

# OOM killer control
docker run -m 512m --oom-kill-disable nginx
docker run -m 512m --oom-score-adj=-500 nginx

# Check container stats
docker stats container_name

CPU Limits

# Limit CPU cores
docker run --cpus=1.5 nginx

# Limit to specific cores
docker run --cpuset-cpus=0,1 nginx

# Limit to specific NUMA node
docker run --cpuset-mems=0 nginx

# CPU shares (relative weight)
docker run --cpu-shares=1024 nginx

# CPU quota and period
docker run --cpu-period=100000 --cpu-quota=50000 nginx
# Same as --cpus=0.5

# Realtime CPU (requires tuning)
docker run --cap-add=sys_nice --cpu-rt-runtime=950000 \
  --ulimit rtprio=99 \
  realtime-container

I/O Limits

# Read/Write IOPS limits
docker run --device-read-iops=/dev/sda:1000 \
  --device-write-iops=/dev/sda:1000 container

# Read/Write bandwidth limits
docker run --device-read-bps=/dev/sda:50mb \
  --device-write-bps=/dev/sda:25mb container

# Blkio weight (cgroup v1)
docker run --blkio-weight=500 container

Monitoring Resources

# Real-time stats
docker stats

# Stats for specific container
docker stats container_name

# JSON format
docker stats --no-stream container_name --format "{{json .}}"

# All containers
docker stats --all

# Inspect limits
docker inspect container_name | grep -i memory
docker inspect container_name | grep -i cpu

58.8 Health Checks

Defining Health Checks

# Dockerfile with HEALTHCHECK
FROM nginx

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD curl -f http://localhost/ || exit 1

# Using wget instead
HEALTHCHECK CMD wget --quiet --tries=1 --spider http://localhost/ || exit 1

# Using custom script
HEALTHCHECK CMD /usr/local/bin/healthcheck.sh

# Docker Compose health check
services:
  web:
    image: nginx
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost/"]
      interval: 30s
      timeout: 3s
      retries: 3
      start_period: 10s

Managing Health Checks

# View health status
docker inspect --format='{{.State.Health}}' container

# Run health check manually
docker healthcheck inspect container

# Check health status in ps
docker ps -a --filter health=healthy

# Disable health check in running container
docker update --health-cmd="none" container

58.9 Multi-Stage Builds

Build Optimization

# Node.js multi-stage build
# Stage 1: Build
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

# Stage 2: Production
FROM node:18-alpine AS production
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
USER node
EXPOSE 3000
CMD ["node", "dist/index.js"]

# Go multi-stage build
FROM golang:1.20 AS builder
WORKDIR /build
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o app

FROM scratch
COPY --from=builder /build/app /app
EXPOSE 8080
CMD ["/app"]

# Python multi-stage
FROM python:3.11-slim AS builder
WORKDIR /build
RUN pip install --user -r requirements.txt

FROM python:3.11-slim
WORKDIR /app
COPY --from=builder /root/.local /root/.local
ENV PATH=/root/.local/bin:$PATH
COPY . .
CMD ["python", "app.py"]

Build Arguments

# Using build arguments
FROM alpine AS builder
ARG VERSION=latest
ARG BUILD_DATE
RUN echo "Building version $VERSION on $BUILD_DATE"

FROM alpine
ARG VERSION
ENV APP_VERSION=$VERSION
CMD ["echo", "Version: $VERSION"]

# Build with args
docker build --build-arg VERSION=1.0.0 \
  --build-arg BUILD_DATE=$(date -u +%Y-%m-%d) \
  -t myapp .

58.10 Docker Swarm

Initializing Swarm

# Initialize swarm
docker swarm init

# Get join token for worker
docker swarm join-token worker

# Get join token for manager
docker swarm join-token manager

# Join as worker
docker swarm join --token SWMTKN-1-... MANAGER_IP:2377

# Leave swarm
docker swarm leave

# Force leave (manager)
docker swarm leave --force

Managing Services

# Create service
docker service create --name myservice --replicas 3 nginx

# List services
docker service ls

# Inspect service
docker service inspect myservice

# Update service
docker service update --replicas 5 myservice
docker service update --image nginx:alpine myservice

# Scale service
docker service scale myservice=10

# Remove service
docker service rm myservice

Managing Stacks

# Deploy stack
docker stack deploy -c docker-compose.yml mystack

# List stacks
docker stack ls

# List services in stack
docker stack services mystack

# List tasks in stack
docker stack ps mystack

# Remove stack
docker stack rm mystack

58.11 Docker Compose

Compose File Basics

version: '3.8'

services:
  web:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./html:/usr/share/nginx/html:ro
    networks:
      - frontend
    depends_on:
      - api
    restart: unless-stopped

  api:
    build: ./api
    environment:
      - DATABASE_URL=postgres://db:5432/app
    networks:
      - frontend
      - backend
    depends_on:
      - db

  db:
    image: postgres:15-alpine
    volumes:
      - db-data:/var/lib/postgresql/data
    networks:
      - backend
    restart: unless-stopped

networks:
  frontend:
  backend:

volumes:
  db-data:

Compose Commands

# Start services
docker-compose up -d

# Start with specific file
docker-compose -f docker-compose.prod.yml up -d

# View logs
docker-compose logs -f

# List containers
docker-compose ps

# Stop services
docker-compose down

# Stop and remove volumes
docker-compose down -v

# Rebuild services
docker-compose build
docker-compose up -d --build

# Scale service
docker-compose up -d --scale web=3

# Execute command in service
docker-compose exec web sh

End of Chapter 58: Docker Complete Guide