Docker_swarm
Chapter 11: Docker Swarm - Container Orchestration
Section titled “Chapter 11: Docker Swarm - Container Orchestration”Table of Contents
Section titled “Table of Contents”- Introduction to Container Orchestration
- What is Docker Swarm?
- Why Docker Swarm?
- When to Use Docker Swarm
- Docker Swarm Architecture
- Setting Up Docker Swarm
- Swarm Services and Stacks
- Scaling and Load Balancing
- Rolling Updates
- Swarm Networking
- Swarm Security
- Best Practices
- Hands-on Lab
Introduction to Container Orchestration
Section titled “Introduction to Container Orchestration”Before diving into Docker Swarm, let’s understand why we need container orchestration in the first place.
The Problem
Section titled “The Problem”Imagine you have a production application running on Docker containers. As your application grows, you face several challenges:
┌─────────────────────────────────────────────────────────────────────────────┐│ SCALING CHALLENGES WITHOUT ORCHESTRATION │├─────────────────────────────────────────────────────────────────────────────┤│ ││ ┌─────────┐ ┌─────────┐ ┌─────────┐ ││ │ Container│ │ Container│ │ Container│ Multiple containers ││ │ App │ │ App │ │ App │ need management ││ └────┬────┘ └────┬────┘ └────┬────┘ ││ │ │ │ ││ └─────────────┼─────────────┘ ││ ▼ ││ ┌───────────────┐ ││ │ Manual Work │ ││ │ - Health check│ ││ │ - Restart │ ││ │ - Scaling │ ││ │ - Networking │ ││ │ - Updates │ ││ └───────────────┘ ││ │└─────────────────────────────────────────────────────────────────────────────┘What is Container Orchestration?
Section titled “What is Container Orchestration?”Container orchestration automates the deployment, management, scaling, and networking of containers. It solves these problems:
- Automatic Healing - Restart failed containers
- Scaling - Scale up/down based on demand
- Load Balancing - Distribute traffic across containers
- Service Discovery - Find services automatically
- Rolling Updates - Update without downtime
- Configuration Management - Manage configurations centrally
┌─────────────────────────────────────────────────────────────────────────────┐│ CONTAINER ORCHESTRATION BENEFITS │├─────────────────────────────────────────────────────────────────────────────┤│ ││ ┌─────────────────────────────────────────────────────────────────────┐ ││ │ ORCHESTRATION LAYER │ ││ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ ││ │ │ Health │ │ Auto │ │ Load │ │ ││ │ │ Monitoring│ │ Scaling │ │ Balancing │ │ ││ │ └─────────────┘ └─────────────┘ └─────────────┘ │ ││ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ ││ │ │ Service │ │ Rolling │ │ Secret │ │ ││ │ │ Discovery │ │ Updates │ │ Management │ │ ││ │ └─────────────┘ └─────────────┘ └─────────────┘ │ ││ └─────────────────────────────────────────────────────────────────────┘ ││ │ ││ ▼ ││ ┌─────────────────────────────────────────────────────────────────────┐ ││ │ CONTAINER CLUSTER │ ││ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ ││ │ │Node 1 │ │Node 2 │ │Node 3 │ │Node 4 │ │ ││ │ │┌───────┐│ │┌───────┐│ │┌───────┐│ │┌───────┐│ │ ││ │ ││Container││ ││Container││ ││Container││ ││Container││ │ ││ │ │└───────┘│ │└───────┘│ │└───────┘│ │└───────┘│ │ ││ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ ││ └─────────────────────────────────────────────────────────────────────┘ ││ │└─────────────────────────────────────────────────────────────────────────────┘What is Docker Swarm?
Section titled “What is Docker Swarm?”Docker Swarm is Docker’s native container orchestration platform. It turns a group of Docker hosts into a single, virtual Docker engine.
Key Definitions
Section titled “Key Definitions”- Swarm: A cluster of Docker hosts organized into a swarm mode
- Node: A Docker instance participating in the swarm
- Manager Node: Controls swarm operations and task distribution
- Worker Node: Executes tasks (containers) assigned by managers
- Service: Definition of tasks to execute on worker nodes
- Task: A single container running on a worker node
- Stack: Collection of services that make up an application
Docker Swarm vs Kubernetes
Section titled “Docker Swarm vs Kubernetes”Many beginners wonder whether to use Docker Swarm or Kubernetes. Here’s a comparison:
┌─────────────────────────────────────────────────────────────────────────────┐│ DOCKER SWARM vs KUBERNETES │├─────────────────────────────────────────────────────────────────────────────┤│ ││ Feature │ Docker Swarm │ Kubernetes ││ ─────────────────────┼─────────────────┼───────────────────────────── ││ Complexity │ Lower │ Higher ││ Learning Curve │ Gentle │ Steep ││ Setup │ Simple │ Complex ││ Scaling │ Easy │ Powerful ││ Enterprise Ready │ Good │ Excellent ││ Cloud Native │ Yes │ Yes ││ Self-Healing │ Yes │ Yes ││ Rolling Updates │ Yes │ Yes ││ Custom Resources │ Limited │ Extensive ││ Community │ Docker │ CNCF ││ ││ Best For: ││ ───────────────────────────────────────────────────────────────────── ││ • Docker Swarm: Small teams, simple needs, Docker-centric ││ • Kubernetes: Large scale, complex deployments, multi-cloud ││ │└─────────────────────────────────────────────────────────────────────────────┘Why Docker Swarm?
Section titled “Why Docker Swarm?”Advantages of Docker Swarm
Section titled “Advantages of Docker Swarm”-
Native Docker Integration
- No additional installation required
- Uses familiar Docker CLI commands
- Seamless integration with Docker Compose
-
Simplicity
- Easy to set up and configure
- Low learning curve
- Perfect for smaller teams
-
Lightweight
- Less resource overhead
- Faster startup times
- Ideal for limited resources
-
Built-in Features
- Load balancing
- Service discovery
- Rolling updates
- Scaling
- Security (TLS)
┌─────────────────────────────────────────────────────────────────────────────┐│ DOCKER SWARM ADVANTAGES │├─────────────────────────────────────────────────────────────────────────────┤│ ││ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ ││ │ SIMPLE SETUP │ │ NATIVE DOCKER │ │ LIGHTWEIGHT │ ││ │ │ │ │ │ │ ││ │ 5 min to get │ │ Same CLI, same │ │ ~50MB overhead │ ││ │ production │ │ API, seamless │ │ No extra daemons│ ││ │ ready │ │ integration │ │ │ ││ └──────────────────┘ └──────────────────┘ └──────────────────┘ ││ ││ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ ││ │ ENTERPRISE │ │ BUILT-IN │ │ SECURE BY │ ││ │ READY │ │ FEATURES │ │ DEFAULT │ ││ │ │ │ │ │ │ ││ │ TLS, RBAC, │ │ LB, SD, Rolling │ │ Mutual TLS, │ ││ │ Secrets, │ │ Updates, Scaling│ │ Encryption │ ││ │ Secrets, │ │ │ │ │ ││ └──────────────────┘ └──────────────────┘ └──────────────────┘ ││ │└─────────────────────────────────────────────────────────────────────────────┘When to Use Docker Swarm
Section titled “When to Use Docker Swarm”Ideal Use Cases
Section titled “Ideal Use Cases”-
Small to Medium Deployments
- 1-100 containers
- Single team or small DevOps team
-
Rapid Prototyping
- Quick deployment needed
- Fast iteration cycles
-
Microservices with Docker
- Multiple small services
- Docker Compose users migrating to production
-
Edge Computing
- Limited resources
- Simple orchestration needs
-
Development/Staging
- Lightweight testing environments
- CI/CD pipeline testing
When NOT to Use Docker Swarm
Section titled “When NOT to Use Docker Swarm”┌─────────────────────────────────────────────────────────────────────────────┐│ WHEN TO CHOOSE OTHER SOLUTIONS │├─────────────────────────────────────────────────────────────────────────────┤│ ││ Choose Kubernetes when: ││ ───────────────────────── ││ • Need multi-cloud or hybrid cloud deployments ││ • Complex custom resource definitions needed ││ • Large-scale production with thousands of containers ││ • Need extensive ecosystem and integrations ││ • Enterprise-grade security and compliance requirements ││ • Need advanced scheduling and placement strategies ││ ││ Choose Docker Swarm when: ││ ───────────────────────── ││ • Starting with container orchestration ││ • Simple microservices architecture ││ • Already using Docker heavily ││ • Quick setup is priority ││ • Limited infrastructure team ││ │└─────────────────────────────────────────────────────────────────────────────┘Docker Swarm Architecture
Section titled “Docker Swarm Architecture”Swarm Cluster Architecture
Section titled “Swarm Cluster Architecture”┌─────────────────────────────────────────────────────────────────────────────┐│ DOCKER SWARM ARCHITECTURE │├─────────────────────────────────────────────────────────────────────────────┤│ ││ ┌─────────────────┐ ││ │ Load Balancer │ ││ │ (External) │ ││ └────────┬────────┘ ││ │ ││ ┌───────────────────────┼───────────────────────┐ ││ │ │ │ ││ ▼ ▼ ▼ ││ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ ││ │ Manager Node │ │ Manager Node │ │ Manager Node │ ││ │ (Leader) │◄────►│ (Member) │◄────►│ (Member) │ ││ │ │ │ │ │ │ ││ │ - Orchestrate │ │ - Replicate │ │ - Replicate │ ││ │ - API endpoint│ │ - Consensus │ │ - Consensus │ ││ │ - Scheduler │ │ - RAFT │ │ - RAFT │ ││ └───────┬───────┘ └───────────────┘ └───────────────┘ ││ │ ││ │ Control Plane (Manager) ││ │ Data Plane (Workers) ││ │ ││ ▼ ││ ┌────────────────────────────────────────────────────────────────┐ ││ │ SWARM CLUSTER │ ││ │ │ ││ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ ││ │ │ Worker Node │ │ Worker Node │ │ Worker Node │ │ ││ │ │ │ │ │ │ │ │ ││ │ │ ┌──────────┐ │ │ ┌──────────┐ │ │ ┌──────────┐ │ │ ││ │ │ │ Task 1 │ │ │ │ Task 2 │ │ │ │ Task 3 │ │ │ ││ │ │ └──────────┘ │ │ └──────────┘ │ │ └──────────┘ │ │ ││ │ │ ┌──────────┐ │ │ ┌──────────┐ │ │ ┌──────────┐ │ │ ││ │ │ │ Task 4 │ │ │ │ Task 5 │ │ │ │ Task 6 │ │ │ ││ │ │ └──────────┘ │ │ └──────────┘ │ │ └──────────┘ │ │ ││ │ └──────────────┘ └──────────────┘ └──────────────┘ │ ││ └────────────────────────────────────────────────────────────────┘ ││ │└─────────────────────────────────────────────────────────────────────────────┘Manager Node Responsibilities
Section titled “Manager Node Responsibilities”-
Cluster Management
- Maintain cluster state
- Schedule tasks
- Handle API requests
-
Raft Consensus
- Leader election
- State replication
- Fault tolerance
-
Orchestration
- Service creation
- Update orchestration
- Health monitoring
Worker Node Responsibilities
Section titled “Worker Node Responsibilities”-
Task Execution
- Run containers
- Report container status
- Execute tasks assigned by managers
-
Networking
- Participate in overlay network
- Route traffic
Setting Up Docker Swarm
Section titled “Setting Up Docker Swarm”Initializing a Swarm
Section titled “Initializing a Swarm”# Initialize Docker Swarm (on the first manager node)docker swarm init --advertise-addr <MANAGER-IP>
# Exampledocker swarm init --advertise-addr 192.168.1.10Adding Worker Nodes
Section titled “Adding Worker Nodes”# Get join token from managerdocker swarm join-token worker
# On worker nodes, run the command provideddocker swarm join --token <WORKER-TOKEN> <MANAGER-IP>:2377
# Exampledocker swarm join --token SWMTKN-1-2abc3defghi4jklmnop5qrstuvwxyz 192.168.1.10:2377Adding Additional Managers
Section titled “Adding Additional Managers”# Get manager join tokendocker swarm join-token manager
# On new manager nodes, run the command provideddocker swarm join --token <MANAGER-TOKEN> <MANAGER-IP>:2377Swarm Management Commands
Section titled “Swarm Management Commands”# Check swarm statusdocker info | grep Swarm
# List nodes in swarmdocker node ls
# List servicesdocker service ls
# Check node detailsdocker node inspect <NODE-ID>
# Leave swarm (from worker)docker swarm leave
# Leave swarm (from manager - forces cluster removal)docker swarm leave --forceVisualizing the Swarm Setup Process
Section titled “Visualizing the Swarm Setup Process”┌─────────────────────────────────────────────────────────────────────────────┐│ SWARM INITIALIZATION FLOW │├─────────────────────────────────────────────────────────────────────────────┤│ ││ Step 1: Initialize Swarm ││ ───────────────────────── ││ ││ ┌─────────────┐ ││ │ Server A │ docker swarm init --advertise-addr 192.168.1.10 ││ │ (Manager) │────────────────┐ ││ └─────────────┘ │ ││ │ ││ ┌─────────────────────────────┴─────────────────────────────┐ ││ │ │ ││ │ • Creates RAFT consensus group │ ││ │ • Generates worker and manager tokens │ ││ │ • Starts swarm mode │ ││ │ • Exposes port 2377 for cluster communication │ ││ │ │ ││ └────────────────────────────────────────────────────────────┘ ││ │ ││ ▼ ││ Step 2: Add Workers ││ ──────────────── ││ ││ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ││ │ Server B │ │ Server C │ │ Server D │ ││ │ (Worker) │ │ (Worker) │ │ (Worker) │ ││ │ │ │ │ │ │ ││ │ docker │ │ docker │ │ docker │ ││ │ swarm join │ │ swarm join │ │ swarm join │ ││ └─────────────┘ └─────────────┘ └─────────────┘ ││ ││ Step 3: Add Manager (Optional) ││ ─────────────────────────────── ││ ││ ┌─────────────┐ ││ │ Server E │ docker swarm join --token manager ... ││ │ (Manager) │ (Requires odd number for quorum) ││ └─────────────┘ ││ │└─────────────────────────────────────────────────────────────────────────────┘Swarm Services and Stacks
Section titled “Swarm Services and Stacks”Creating a Service
Section titled “Creating a Service”# Create a simple servicedocker service create --name my-web --replicas 3 -p 8080:80 nginx
# Create service with environment variablesdocker service create --name my-app \ --replicas 5 \ -e NODE_ENV=production \ -e DB_HOST=postgres \ myregistry.io/my-app:latest
# Create service with constraints (run on specific nodes)docker service create --name database \ --constraint node.labels.disk==ssd \ postgres:15
# Create service with resource limitsdocker service create --name api \ --limit-memory 512M \ --limit-cpu 0.5 \ myregistry.io/api:latestService Management
Section titled “Service Management”# List servicesdocker service ls
# Inspect servicedocker service inspect my-web
# View service logsdocker service logs my-web
# Update servicedocker service update --replicas 5 my-web
# Remove servicedocker service rm my-webScaling Services
Section titled “Scaling Services”# Scale service updocker service scale my-web=10
# Scale multiple servicesdocker service scale my-web=5 my-api=3 my-worker=2Understanding Services
Section titled “Understanding Services”┌─────────────────────────────────────────────────────────────────────────────┐│ SERVICE AND TASK RELATIONSHIP │├─────────────────────────────────────────────────────────────────────────────┤│ ││ Service Definition ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ name: my-web │ ││ │ image: nginx:latest │ ││ │ replicas: 3 │ ││ │ ports: 8080:80 │ ││ │ update_config: │ ││ │ parallelism: 1 │ ││ │ delay: 10s │ ││ └─────────────────────────────────────────────────────────────────┘ ││ │ ││ ▼ ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ SERVICE │ ││ │ │ ││ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ ││ │ │ Task │ │ Task │ │ Task │ ← Container Instance │ ││ │ │ 1 │ │ 2 │ │ 3 │ │ ││ │ └────┬────┘ └────┬────┘ └────┬────┘ │ ││ │ │ │ │ │ ││ │ ┌────┴────────────┴────────────┴────┐ │ ││ │ │ Task Definition │ │ ││ │ │ • Container spec │ │ ││ │ │ • Resource limits │ │ ││ │ │ • Restart policy │ │ ││ │ │ • Placement constraints │ │ ││ │ └─────────────────────────────────────┘ │ ││ │ │ ││ └─────────────────────────────────────────────────────────────────┘ ││ ││ Task State Machine: ││ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ││ │ new │──►│pending │──►│assigned│──►│accepted│──►│running │ ││ └────────┘ └────────┘ └────────┘ └────────┘ └────────┘ ││ │ │ │ ││ ▼ ▼ ▼ ││ ┌────────┐ ┌────────┐ ┌────────┐ ││ │rejectd │ │prepare │ │failed │ ││ └────────┘ └────────┘ └────────┘ ││ │└─────────────────────────────────────────────────────────────────────────────┘Using Docker Compose with Swarm
Section titled “Using Docker Compose with Swarm”version: '3.8'
services: web: image: nginx:latest ports: - "80:80" deploy: replicas: 3 update_config: parallelism: 1 delay: 10s restart_policy: condition: on-failure max_attempts: 3
api: image: myapi:latest environment: - DB_HOST=postgres deploy: replicas: 2 resources: limits: cpus: '0.5' memory: 512M
postgres: image: postgres:15 volumes: - db-data:/var/lib/postgresql/data deploy: replicas: 1
volumes: db-data:# Deploy stackdocker stack deploy -c docker-compose.yml myapp
# List stacksdocker stack ls
# List services in stackdocker stack services myapp
# Remove stackdocker stack rm myappScaling and Load Balancing
Section titled “Scaling and Load Balancing”Scaling Services
Section titled “Scaling Services”# Manual scalingdocker service scale myapp-web=5
# Auto-scaling (using docker-compose with deploy mode)docker service update --replicas 5 myapp-webLoad Balancing in Swarm
Section titled “Load Balancing in Swarm”┌─────────────────────────────────────────────────────────────────────────────┐│ DOCKER SWARM LOAD BALANCING │├─────────────────────────────────────────────────────────────────────────────┤│ ││ ┌─────────────────┐ ││ │ External LB │ ││ │ (HAProxy, │ ││ │ Cloud LB) │ ││ └────────┬────────┘ ││ │ ││ │ Incoming Traffic ││ ▼ ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ SWARM CLUSTER │ ││ │ │ ││ │ ┌─────────────────────────────────────────────────────────┐ │ ││ │ │ Internal Load Balancer (Mesh) │ │ ││ │ │ │ │ ││ │ │ ┌─────────────────────────────────────────────────────┐ │ │ ││ │ │ │ Virtual IP (VIP) / DNS Resolution │ │ │ ││ │ │ │ │ │ │ ││ │ │ │ Service: myapp-web (replicas: 3) │ │ │ ││ │ │ │ │ │ │ ││ │ │ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │ │ ││ │ │ │ │Node:1 │ │Node:2 │ │Node:3 │ │ │ │ ││ │ │ │ │Task:1 │ │Task:2 │ │Task:3 │ │ │ │ ││ │ │ │ │:8080 │ │:8080 │ │:8080 │ │ │ │ ││ │ │ │ └─────────┘ └─────────┘ └─────────┘ │ │ │ ││ │ │ │ │ │ │ ││ │ │ └─────────────────────────────────────────────────────┘ │ │ ││ │ │ │ │ ││ │ └─────────────────────────────────────────────────────────┘ │ ││ │ │ ││ └─────────────────────────────────────────────────────────────────┘ ││ ││ Routing Mesh Features: ││ • Automatic load balancing across all containers ││ • Service discovery via DNS ││ • External access via published ports ││ • Internal communication via service names ││ │└─────────────────────────────────────────────────────────────────────────────┘Service Discovery
Section titled “Service Discovery”# Services can be reached by name# From any container in the swarm:curl http://my-web-service
# Or using DNSping my-web-service
# Inspect service endpointdocker service inspect --format '{{json .Endpoint}}' my-web-serviceRolling Updates
Section titled “Rolling Updates”Performing Rolling Updates
Section titled “Performing Rolling Updates”# Update service imagedocker service update --image myapp:v2.0 myapp
# Update with custom update configurationdocker service update \ --image myapp:v2.0 \ --update-parallelism 2 \ --update-delay 15s \ --update-failure-action rollback \ myapp
# Rollback to previous versiondocker service rollback myappUpdate Configuration
Section titled “Update Configuration”# docker-compose.yml with update configservices: web: image: myapp:latest deploy: replicas: 3 update_config: parallelism: 2 # Update 2 containers at a time delay: 10s # Wait 10s between updates failure_action: rollback # Rollback on failure monitor: 5s # Monitor for 5s after update max_failure_ratio: 0.3 # Allow 30% failure rollback_config: parallelism: 1 delay: 5s failure_action: pauseVisualizing Rolling Updates
Section titled “Visualizing Rolling Updates”┌─────────────────────────────────────────────────────────────────────────────┐│ ROLLING UPDATE PROCESS │├─────────────────────────────────────────────────────────────────────────────┤│ ││ Initial State: replicas=3, image=v1.0 ││ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ││ │ [v1.0] │ │ [v1.0] │ │ [v1.0] │ ││ │ Running │ │ Running │ │ Running │ ││ └─────────────┘ └─────────────┘ └─────────────┘ ││ ││ ───────────────────────────────────────────────────────────────────── ││ ││ Step 1: Update Task 1 (parallelism=1, delay=10s) ││ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ││ │ [v2.0] │ │ [v1.0] │ │ [v1.0] │ ││ │ Updating ⏳ │ │ Running │ │ Running │ ││ └─────────────┘ └─────────────┘ └─────────────┘ ││ ││ Wait 10s... ││ ││ Step 2: Update Task 2 ││ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ││ │ [v2.0] │ │ [v2.0] │ │ [v1.0] │ ││ │ Running ✓ │ │ Updating ⏳ │ │ Running │ ││ └─────────────┘ └─────────────┘ └─────────────┘ ││ ││ Wait 10s... ││ ││ Step 3: Update Task 3 ││ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ││ │ [v2.0] │ │ [v2.0] │ │ [v2.0] │ ││ │ Running ✓ │ │ Running ✓ │ │ Running ✓ │ ││ └─────────────┘ └─────────────┘ └─────────────┘ ││ ││ ───────────────────────────────────────────────────────────────────── ││ ││ Final State: replicas=3, image=v2.0 ││ ││ ROLLBACK PROCESS: ││ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ││ │ [v2.0] │ │ [v2.0] │ │ [v2.0] │ ││ │ Running ✓ │──►│ Rollingback │──►│ [v1.0] │ ││ └─────────────┘ └─────────────┘ └─────────────┘ ││ │└─────────────────────────────────────────────────────────────────────────────┘Swarm Networking
Section titled “Swarm Networking”Overlay Network
Section titled “Overlay Network”# Create an overlay networkdocker network create \ --driver overlay \ --attachable \ my-overlay-network
# Use overlay network in servicedocker service create \ --name my-service \ --network my-overlay-network \ nginxIngress Network
Section titled “Ingress Network”┌─────────────────────────────────────────────────────────────────────────────┐│ SWARM INGRESS NETWORK │├─────────────────────────────────────────────────────────────────────────────┤│ ││ External Traffic ││ │ ││ ▼ ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ INGRESS NETWORK (Overlay) │ ││ │ │ ││ │ ┌─────────────────────────────────────────────────────────┐ │ ││ │ │ Routing Mesh │ │ ││ │ │ • Distributes incoming traffic to any node │ │ ││ │ │ • Uses IPVS for load balancing │ │ ││ │ │ • Works at OSI Layer 3 (IP) │ │ ││ │ └─────────────────────────────────────────────────────────┘ │ ││ │ │ │ ││ │ ┌───────────────────────────┼───────────────────────────────┐ │ ││ │ │ │ │ │ ││ │ ▼ ▼ ▼ │ ││ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ││ │ │Node 1 │ │Node 2 │ │Node 3 │ ││ │ │ │ │ │ │ │ ││ │ │ ┌─────┐ │ │ ┌─────┐ │ │ ┌─────┐ │ ││ │ │ │Task │ │ │ │Task │ │ │ │Task │ │ ││ │ │ │ :80 │ │ │ │ :80 │ │ │ │ :80 │ │ ││ │ │ └─────┘ │ │ └─────┘ │ │ └─────┘ │ ││ │ └─────────┘ └─────────┘ └─────────┘ ││ │ │ ││ └─────────────────────────────────────────────────────────────────┘ ││ ││ Traffic to any node:8080 routes to the correct container ││ │└─────────────────────────────────────────────────────────────────────────────┘Network Commands
Section titled “Network Commands”# List networksdocker network ls
# Inspect networkdocker network inspect my-overlay-network
# Remove networkdocker network rm my-networkSwarm Security
Section titled “Swarm Security”Swarm Security Features
Section titled “Swarm Security Features”┌─────────────────────────────────────────────────────────────────────────────┐│ DOCKER SWARM SECURITY LAYERS │├─────────────────────────────────────────────────────────────────────────────┤│ ││ ┌─────────────────────────────────────────────────────────────────────┐ ││ │ SWARM SECURITY │ ││ │ │ ││ │ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ ││ │ │ MUTUAL TLS │ │ ENCRYPTED │ │ RBAC │ │ ││ │ │ │ │ NETWORKS │ │ │ │ ││ │ │ • Node auth │ │ │ │ • Roles │ │ ││ │ │ • Encrypted │ │ • Data plane │ │ • Policies │ │ ││ │ │ comms │ │ encryption │ │ • Grants │ │ ││ │ │ • Certificate │ │ • TLS for │ │ │ │ ││ │ │ rotation │ │ ingress │ │ │ │ ││ │ └───────────────┘ └───────────────┘ └───────────────┘ │ ││ │ │ ││ │ ┌───────────────┐ ┌───────────────┐ │ ││ │ │ SECRETS │ │ NODE │ │ ││ │ │ │ │ LOCKING │ │ ││ │ │ • Encrypted │ │ │ │ ││ │ │ at rest │ │ • Rotate │ │ ││ │ │ • TLS certs │ │ keys │ │ ││ │ │ • Passwords │ │ • Auto-lock │ │ ││ │ │ • Keys │ │ │ │ ││ │ └───────────────┘ └───────────────┘ │ ││ │ │ ││ └─────────────────────────────────────────────────────────────────────┘ ││ │└─────────────────────────────────────────────────────────────────────────────┘Secrets Management
Section titled “Secrets Management”# Create a secretecho "my-password" | docker secret create db_password -
# Use secret in servicedocker service create \ --name my-app \ --secret db_password \ myapp:latest
# Use secret as environment variabledocker service create \ --name my-app \ --secret source=db_password,target=DB_PASSWORD,mode=0400 \ myapp:latestEnabling Auto-Lock
Section titled “Enabling Auto-Lock”# Initialize swarm with auto-lockdocker swarm init --autolock
# Or enable on existing swarmdocker swarm update --autolock=true
# Unlock with keydocker swarm unlock
# View unlock keydocker swarm join-token --rotate managerBest Practices
Section titled “Best Practices”Swarm Best Practices
Section titled “Swarm Best Practices”┌─────────────────────────────────────────────────────────────────────────────┐│ DOCKER SWARM BEST PRACTICES │├─────────────────────────────────────────────────────────────────────────────┤│ ││ 1. Manager Nodes ││ ─────────────── ││ • Use 3 or 5 managers (odd number for quorum) ││ • Don't run workloads on managers (use constraints) ││ • Distribute managers across availability zones ││ • Enable auto-lock for additional security ││ ││ 2. Worker Nodes ││ ───────────── ││ • Minimum 3 workers for high availability ││ • Use similar hardware across nodes ││ • Separate nodes by workload type (compute, memory, I/O) ││ ││ 3. Services ││ ───────── ││ • Use health checks in container definitions ││ • Set appropriate replica counts ││ • Use update delay for gradual rollouts ││ • Always set resource limits ││ ││ 4. Networking ││ ─────────── ││ • Use custom overlay networks for isolation ││ • Enable encryption on sensitive networks ││ • Limit port exposure to necessary ports only ││ ││ 5. Security ││ ───────── ││ • Use secrets for sensitive data ││ • Scan images for vulnerabilities ││ • Use read-only file systems when possible ││ • Run containers as non-root users ││ │└─────────────────────────────────────────────────────────────────────────────┘Resource Limits
Section titled “Resource Limits”# Service with resource limitsdocker service create \ --name api \ --limit-memory 512M \ --limit-cpu 0.5 \ --reserve-memory 256M \ --reserve-cpu 0.25 \ myapi:latestHealth Checks
Section titled “Health Checks”# Service with health checkdocker service create \ --name web \ --health-cmd "curl -f http://localhost/ || exit 1" \ --health-interval 30s \ --health-timeout 10s \ --health-retries 3 \ --health-start-period 30s \ nginxHands-on Lab
Section titled “Hands-on Lab”Lab: Deploy a Multi-Service Application
Section titled “Lab: Deploy a Multi-Service Application”In this hands-on lab, we’ll deploy a complete application stack using Docker Swarm.
Prerequisites
Section titled “Prerequisites”- 3 or more Docker hosts (can be VMs or cloud instances)
- Docker Engine 19.03 or later
Lab Steps
Section titled “Lab Steps”# Step 1: Initialize swarm on first nodedocker swarm init --advertise-addr <MANAGER-IP>
# Step 2: Get join tokensdocker swarm join-token workerdocker swarm join-token manager
# Step 3: Join worker nodes (on each worker)docker swarm join --token <WORKER-TOKEN> <MANAGER-IP>:2377
# Step 4: Verify clusterdocker node ls
# Step 5: Create overlay networkdocker network create --driver overlay myapp-network
# Step 6: Deploy stackdocker stack deploy -c <<EOFversion: '3.8'
services: redis: image: redis:alpine networks: - myapp-network deploy: replicas: 1
web: image: nginx:alpine ports: - "80:80" networks: - myapp-network deploy: replicas: 2 update_config: parallelism: 1 delay: 10s restart_policy: condition: on-failure
worker: image: myapp-worker:latest networks: - myapp-network deploy: replicas: 3
networks: myapp-network: external: trueEOFmyapp
# Step 7: Monitor deploymentdocker stack services myappdocker service ps myapp_web
# Step 8: Scale the applicationdocker service scale myapp_web=5docker service scale myapp_worker=10
# Step 9: Update the applicationdocker service update --image nginx:latest myapp_web
# Step 10: Check logsdocker service logs myapp_web
# Step 11: Clean updocker stack rm myappdocker network rm myapp-networkSummary
Section titled “Summary”In this chapter, we’ve covered:
- Container Orchestration - Why we need orchestration for production workloads
- Docker Swarm - Native Docker container orchestration platform
- Architecture - Manager nodes, worker nodes, and their roles
- Setup - Initializing and configuring a swarm cluster
- Services and Stacks - Deploying applications as services and stacks
- Scaling & Load Balancing - Horizontal scaling and built-in load balancing
- Rolling Updates - Performing zero-downtime updates
- Networking - Overlay networks and routing mesh
- Security - TLS, secrets, and RBAC
- Best Practices - Production-ready configurations
Next Steps
Section titled “Next Steps”In the next chapter, we’ll dive into Docker Security (Chapter 12), covering:
- Container security best practices
- Seccomp and AppArmor profiles
- Docker content trust
- Security scanning