Terraform on AWS
Chapter 41: Terraform on AWS
Section titled “Chapter 41: Terraform on AWS”Infrastructure as Code with Terraform
Section titled “Infrastructure as Code with Terraform”41.1 Overview
Section titled “41.1 Overview”Terraform is an open-source Infrastructure as Code (IaC) tool that enables you to define and provision infrastructure using a high-level configuration language.
Terraform Overview+------------------------------------------------------------------+| || +------------------------+ || | Terraform | || +------------------------+ || | || +---------------------+---------------------+ || | | | | || v v v v || +----------+ +----------+ +----------+ +----------+ || | HCL | | State | | Providers| | Modules | || | Config | | Management| | | | | || | | | | | | | | || | - Declar | | - Track | | - AWS | | - Reuse | || | ative | | Infra | | - Azure | | - Share | || | - Idempo | | - Lock | | - GCP | | - Version| || +----------+ +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+Key Features
Section titled “Key Features”| Feature | Description |
|---|---|
| HCL | HashiCorp Configuration Language |
| State | Tracks infrastructure state |
| Providers | Cloud platform integrations |
| Modules | Reusable configurations |
41.2 Terraform Configuration
Section titled “41.2 Terraform Configuration”Basic Structure
Section titled “Basic Structure” Terraform Configuration Structure+------------------------------------------------------------------+| || main.tf || +------------------------------------------------------------+ || | | || | # Provider configuration | || | provider "aws" { | || | region = "us-east-1" | || | } | || | | || | # Resource definition | || | resource "aws_vpc" "main" { | || | cidr_block = "10.0.0.0/16" | || | tags = { | || | Name = "main-vpc" | || | } | || | } | || | | || +------------------------------------------------------------+ || || variables.tf || +------------------------------------------------------------+ || | | || | variable "region" { | || | type = string | || | default = "us-east-1" | || | description = "AWS region" | || | } | || | | || +------------------------------------------------------------+ || || outputs.tf || +------------------------------------------------------------+ || | | || | output "vpc_id" { | || | value = aws_vpc.main.id | || | description = "VPC ID" | || | } | || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Resource Syntax
Section titled “Resource Syntax”# Basic resource syntaxresource "aws_instance" "web_server" { ami = "ami-0c55b159cbfafe1f0" instance_type = "t3.micro"
tags = { Name = "WebServer" Environment = "Production" }}
# Resource with dependenciesresource "aws_eip" "web_eip" { instance = aws_instance.web_server.id vpc = true}
# Resource with countresource "aws_instance" "server" { count = 3 ami = "ami-0c55b159cbfafe1f0" instance_type = "t3.micro"
tags = { Name = "Server-${count.index + 1}" }}
# Resource with for_eachresource "aws_instance" "server" { for_each = toset(["web", "api", "db"]) ami = "ami-0c55b159cbfafe1f0" instance_type = "t3.micro"
tags = { Name = "Server-${each.key}" }}41.3 Terraform State
Section titled “41.3 Terraform State”State Management
Section titled “State Management” Terraform State Management+------------------------------------------------------------------+| || Local State || +------------------------------------------------------------+ || | | || | terraform.tfstate | || | +--------------------------------------------------------+ | || | | - Stored locally | | || | | - Not recommended for teams | | || | | - No locking | | || | +--------------------------------------------------------+ | || | | || +------------------------------------------------------------+ || || Remote State (S3 + DynamoDB) || +------------------------------------------------------------+ || | | || | terraform { | || | backend "s3" { | || | bucket = "my-terraform-state" | || | key = "prod/terraform.tfstate" | || | region = "us-east-1" | || | encrypt = true | || | dynamodb_table = "terraform-locks" | || | } | || | } | || | | || +------------------------------------------------------------+ || || State Locking || +------------------------------------------------------------+ || | | || | - Prevents concurrent modifications | || | - DynamoDB table for S3 backend | || | - Automatic lock acquisition | || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+41.4 Terraform Modules
Section titled “41.4 Terraform Modules”Module Structure
Section titled “Module Structure” Terraform Module Structure+------------------------------------------------------------------+| || modules/ || +------------------------------------------------------------+ || | | || | vpc/ | || | +--------------------------------------------------------+ | || | | main.tf # Main resource definitions | | || | | variables.tf # Input variables | | || | | outputs.tf # Output values | | || | | README.md # Module documentation | | || | +--------------------------------------------------------+ | || | | || +------------------------------------------------------------+ || || Using Modules || +------------------------------------------------------------+ || | | || | module "vpc" { | || | source = "./modules/vpc" | || | | || | vpc_cidr = "10.0.0.0/16" | || | environment = "production" | || | az_count = 3 | || | } | || | | || | # Using remote module from registry | || | module "vpc" { | || | source = "terraform-aws-modules/vpc/aws" | || | version = "~> 3.0" | || | | || | name = "my-vpc" | || | cidr = "10.0.0.0/16" | || | } | || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+41.5 Terraform Workflow
Section titled “41.5 Terraform Workflow”Core Workflow
Section titled “Core Workflow” Terraform Workflow+------------------------------------------------------------------+| || 1. Write || +------------------------------------------------------------+ || | | || | - Define infrastructure in HCL | || | - Create configuration files | || | | || +------------------------------------------------------------+ || | || v || 2. Plan || +------------------------------------------------------------+ || | | || | $ terraform plan | || | | || | - Preview changes | || | - Compare state with configuration | || | - Show execution plan | || | | || +------------------------------------------------------------+ || | || v || 3. Apply || +------------------------------------------------------------+ || | | || | $ terraform apply | || | | || | - Execute changes | || | - Update state | || | - Provision infrastructure | || | | || +------------------------------------------------------------+ || | || v || 4. Destroy || +------------------------------------------------------------+ || | | || | $ terraform destroy | || | | || | - Remove all resources | || | - Clean up state | || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+41.6 CLI Commands
Section titled “41.6 CLI Commands”# Initialize Terraformterraform init
# Initialize with backend configurationterraform init -backend-config="bucket=my-state"
# Validate configurationterraform validate
# Format configuration filesterraform fmt
# Plan changesterraform plan
# Plan and save to fileterraform plan -out=tfplan
# Apply changesterraform apply
# Apply saved planterraform apply tfplan
# Apply with auto-approveterraform apply -auto-approve
# Destroy infrastructureterraform destroy
# Show stateterraform show
# List resources in stateterraform state list
# Show specific resourceterraform state show aws_vpc.main
# Move resource in stateterraform state mv aws_vpc.main aws_vpc.primary
# Remove resource from stateterraform state rm aws_vpc.main
# Import existing resourceterraform import aws_vpc.main vpc-12345678
# Output valuesterraform output
# Get specific outputterraform output vpc_id
# Workspace managementterraform workspace listterraform workspace new devterraform workspace select devterraform workspace delete dev
# Refresh stateterraform refresh
# Graph dependenciesterraform graph | dot -Tpng > graph.png41.7 Advanced Terraform Features
Section titled “41.7 Advanced Terraform Features”Terraform Functions
Section titled “Terraform Functions” Terraform Built-in Functions+------------------------------------------------------------------+| || String Functions || +------------------------------------------------------------+ || | join(", ", ["a", "b", "c"]) # "a, b, c" | || | split(", ", "a, b, c") # ["a", "b", "c"] | || | lower("HELLO") # "hello" | || | upper("hello") # "HELLO" | || | substr("hello world", 0, 5) # "hello" | || | replace("hello", "l", "L") # "heLLo" | || | format("Hello %s!", "World") # "Hello World!" | || +------------------------------------------------------------+ || || Numeric Functions || +------------------------------------------------------------+ || | max(1, 2, 3) # 3 | || | min(1, 2, 3) # 1 | || | sum([1, 2, 3]) # 6 | || | abs(-5) # 5 | || | ceil(4.3) # 5 | || | floor(4.7) # 4 | || +------------------------------------------------------------+ || || Collection Functions || +------------------------------------------------------------+ || | length(["a", "b"]) # 2 | || | element(["a", "b", "c"], 1) # "b" | || | contains(["a", "b"], "a") # true | || | distinct(["a", "a", "b"]) # ["a", "b"] | || | merge({a="1"}, {b="2"}) # {a="1", b="2"} | || | keys({a=1, b=2}) # ["a", "b"] | || | values({a=1, b=2}) # [1, 2] | || +------------------------------------------------------------+ || || Encoding Functions || +------------------------------------------------------------+ || | base64encode("hello") # "aGVsbG8=" | || | base64decode("aGVsbG8=") # "hello" | || | jsonencode({"hello"="world"}) # {"hello":"world"} | || | jsondecode("{\"hello\":\"world\"}") # {"hello"="world"} | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Conditional Expressions and Loops
Section titled “Conditional Expressions and Loops”# Conditional expression (ternary)resource "aws_instance" "web" { count = var.environment == "production" ? 3 : 1 instance_type = var.environment == "production" ? "t3.large" : "t3.micro"
tags = { Name = var.enable_monitoring ? "Monitored-Server" : "Server" }}
# Dynamic blocks for repeated nested blocksresource "aws_security_group" "example" { name = "dynamic-sg"
dynamic "ingress" { for_each = var.ingress_rules content { from_port = ingress.value.from_port to_port = ingress.value.to_port protocol = ingress.value.protocol cidr_blocks = ingress.value.cidr_blocks } }}
# For expressionslocals { # Transform list uppercase_names = [for name in var.names : upper(name)]
# Filter list long_names = [for name in var.names : name if length(name) > 5]
# Transform map instance_types = { for k, v in var.environments : k => v.instance_type }
# Create map from list name_to_id = { for instance in aws_instance.web : instance.tags.Name => instance.id }}Terraform Data Sources
Section titled “Terraform Data Sources” Terraform Data Sources+------------------------------------------------------------------+| || Data sources read existing AWS resources || +------------------------------------------------------------+ || | | || | # Get latest Amazon Linux 2 AMI | || | data "aws_ami" "amazon_linux" { | || | most_recent = true | || | owners = ["amazon"] | || | | || | filter { | || | name = "name" | || | values = ["amzn2-ami-hvm-*-x86_64-gp2"] | || | } | || | } | || | | || | # Get default VPC | || | data "aws_vpc" "default" { | || | default = true | || | } | || | | || | # Get subnet IDs in VPC | || | data "aws_subnets" "default" { | || | filter { | || | name = "vpc-id" | || | values = [data.aws_vpc.default.id] | || | } | || | } | || | | || | # Use data source in resource | || | resource "aws_instance" "web" { | || | ami = data.aws_ami.amazon_linux.id | || | instance_type = "t3.micro" | || | subnet_id = data.aws_subnets.default.ids[0] | || | } | || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Terraform Locals
Section titled “Terraform Locals”# Local values for DRY codelocals { common_tags = { Environment = var.environment Project = var.project_name ManagedBy = "Terraform" CreatedAt = timestamp() }
# Computed values name_prefix = "${var.project_name}-${var.environment}"
# Conditional locals instance_count = var.environment == "production" ? 3 : 1
# Complex transformations subnet_cidrs = [for i in range(3) : cidrsubnet(var.vpc_cidr, 8, i)]}
# Using locals in resourcesresource "aws_instance" "web" { count = local.instance_count ami = var.ami_id instance_type = "t3.micro"
tags = merge(local.common_tags, { Name = "${local.name_prefix}-web-${count.index + 1}" })}Terraform Lifecycle Rules
Section titled “Terraform Lifecycle Rules”resource "aws_instance" "web" { ami = var.ami_id instance_type = "t3.micro"
# Lifecycle management lifecycle { # Prevent destruction prevent_destroy = true
# Create before destroy (for zero-downtime updates) create_before_destroy = true
# Ignore changes to specific attributes ignore_changes = [ ami, # Ignore AMI changes tags["LastUpdated"] # Ignore specific tag changes ]
# Replace resource when condition changes replace_triggered_by = [ aws_security_group.web.id ] }}
# Blue-green deployment exampleresource "aws_launch_template" "web" { name_prefix = "web-" image_id = var.ami_id instance_type = "t3.micro"
lifecycle { create_before_destroy = true }}41.8 Terraform Cloud & Enterprise
Section titled “41.8 Terraform Cloud & Enterprise”Terraform Cloud Features
Section titled “Terraform Cloud Features” Terraform Cloud Architecture+------------------------------------------------------------------+| || +------------------------+ || | Terraform Cloud | || +------------------------+ || | || +---------------------+---------------------+ || | | | | || v v v v || +----------+ +----------+ +----------+ +----------+ || | Remote | | State | | Policy | | Private | || | Execution| | Management| | as Code | | Module | || | | | | | | | Registry| || | - VCS | | - Encrypted| | - Sentinel| | - Share | || | Integra-| | - Versioning| | - OPA | | - Version| || | tion | | - Locking| | - Cost | | - Access | || | - API | | - History| | Estim- | | - Control| || | - CLI | | | | ation | | | || +----------+ +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+Terraform Cloud Configuration
Section titled “Terraform Cloud Configuration”# Configure Terraform Cloud backendterraform { cloud { organization = "my-organization"
workspaces { name = "my-workspace" } }}
# Workspace-based configuration# terraform.tfvars in Terraform Cloudenvironment = "production"region = "us-east-1"
# Environment variables in Terraform Cloud# TF_VAR_database_password = "sensitive-value"# AWS_ACCESS_KEY_ID = "AKIA..."# AWS_SECRET_ACCESS_KEY = "secret..."41.9 CI/CD Integration with Terraform
Section titled “41.9 CI/CD Integration with Terraform”GitHub Actions Integration
Section titled “GitHub Actions Integration”name: Terraform CI/CD
on: push: branches: [main] pull_request: branches: [main]
jobs: terraform: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v3
- name: Setup Terraform uses: hashicorp/setup-terraform@v2 with: terraform_version: 1.5.0
- name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v2 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: us-east-1
- name: Terraform Init run: terraform init
- name: Terraform Format run: terraform fmt -check
- name: Terraform Validate run: terraform validate
- name: Terraform Plan run: terraform plan -out=tfplan if: github.event_name == 'pull_request'
- name: Terraform Apply run: terraform apply -auto-approve if: github.ref == 'refs/heads/main' && github.event_name == 'push'41.10 Best Practices
Section titled “41.10 Best Practices”Terraform Best Practices
Section titled “Terraform Best Practices” Terraform Best Practices+------------------------------------------------------------------+| || 1. Use remote state with locking || +------------------------------------------------------------+ || | - S3 backend with DynamoDB for locking | || | - Enable encryption | || +------------------------------------------------------------+ || || 2. Use modules for reusability || +------------------------------------------------------------+ || | - Create reusable modules | || | - Use Terraform Registry modules | || +------------------------------------------------------------+ || || 3. Use variables for flexibility || +------------------------------------------------------------+ || | - Parameterize configurations | || | - Use variable files (terraform.tfvars) | || +------------------------------------------------------------+ || || 4. Implement proper naming || +------------------------------------------------------------+ || | - Consistent resource naming | || | - Use tags for organization | || +------------------------------------------------------------+ || || 5. Use workspaces for environments || +------------------------------------------------------------+ || | - Separate state per environment | || | - dev, staging, prod workspaces | || +------------------------------------------------------------+ || || 6. Version control your code || +------------------------------------------------------------+ || | - Store in Git | || | - Use branches for changes | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+41.11 Why This Matters in DevOps/SRE
Section titled “41.11 Why This Matters in DevOps/SRE”Terraform is the de facto standard for Infrastructure as Code. It’s the tool SREs use to define, version, and manage entire cloud environments declaratively.
Terraform in DevOps/SRE+------------------------------------------------------------------+| || Why Terraform is Critical: || || 1. Infrastructure Reproducibility || +----------------------------------------------------------+ || | - Same config → same infrastructure every time | || | - Disaster recovery: rebuild entire env from code | || | - Environment parity: dev mirrors prod | || +----------------------------------------------------------+ || || 2. Change Management & Audit || +----------------------------------------------------------+ || | - Git history = infrastructure change log | || | - PR reviews for infrastructure changes | || | - terraform plan in CI = change impact preview | || +----------------------------------------------------------+ || || 3. Team Collaboration || +----------------------------------------------------------+ || | - Remote state with locking prevents conflicts | || | - Modules standardize patterns across teams | || | - Workspaces separate environments cleanly | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+41.12 Linux Systems Perspective
Section titled “41.12 Linux Systems Perspective”Terraform Development from Arch Linux
Section titled “Terraform Development from Arch Linux”# Install Terraform on Arch Linuxsudo pacman -S terraform# Or specific version via tfenvyay -S tfenvtfenv install 1.7.0tfenv use 1.7.0
# Essential Terraform aliases for ~/.zshrcalias tf="terraform"alias tfp="terraform plan"alias tfa="terraform apply"alias tfd="terraform destroy"alias tfi="terraform init"alias tfv="terraform validate"alias tff="terraform fmt -recursive"alias tfs="terraform state list"
# Pre-commit hooks for Terraform# Install pre-commitsudo pacman -S python-pre-commit
cat > .pre-commit-config.yaml << 'EOF'repos: - repo: https://github.com/antonbabenko/pre-commit-terraform rev: v1.88.0 hooks: - id: terraform_fmt - id: terraform_validate - id: terraform_tflint - id: terraform_docsEOFpre-commit install
# Terraform state recovery scripttf-unlock() { local lock_id="$1" terraform force-unlock -force "$lock_id"}
# Quick drift detectiontf-drift() { terraform plan -detailed-exitcode 2>&1 local exit_code=$? case $exit_code in 0) echo "✅ No drift detected" ;; 1) echo "❌ Error running plan" ;; 2) echo "⚠️ DRIFT DETECTED - infrastructure has changed outside Terraform" ;; esac}41.13 Troubleshooting Guide
Section titled “41.13 Troubleshooting Guide”| Issue | Cause | Solution |
|---|---|---|
| State lock stuck | Previous apply crashed | terraform force-unlock <LOCK_ID> |
| Provider version conflict | Incompatible constraints | Pin versions in required_providers block |
Error: Cycle detected | Circular resource dependencies | Restructure resources, use depends_on explicitly |
| State file corruption | Manual edits or concurrent writes | Restore from S3 versioned state backup |
Error: Unsupported attribute | Resource schema changed | Run terraform init -upgrade to update providers |
| Drift between state and reality | Manual console changes | Run terraform import or terraform refresh |
# Debug Terraform issues# Enable debug loggingexport TF_LOG=DEBUGexport TF_LOG_PATH=./terraform-debug.logterraform plan
# Check state for specific resourceterraform state show aws_instance.web
# List all resources in stateterraform state list | grep "aws_instance"
# Remove orphaned resource from stateterraform state rm aws_instance.deleted_manually41.14 Common Mistakes & Anti-Patterns
Section titled “41.14 Common Mistakes & Anti-Patterns” Terraform Anti-Patterns+------------------------------------------------------------------+| || ❌ Mistake 1: Local State in Production || +----------------------------------------------------------+ || | Problem: State file on one person's laptop | || | Impact: No team collaboration, data loss risk | || | Fix: Always use S3+DynamoDB remote backend | || +----------------------------------------------------------+ || || ❌ Mistake 2: Committing State to Git || +----------------------------------------------------------+ || | Problem: State contains secrets (passwords, keys) | || | Impact: Credential exposure in version control | || | Fix: Add terraform.tfstate* to .gitignore | || +----------------------------------------------------------+ || || ❌ Mistake 3: One Giant State File || +----------------------------------------------------------+ || | Problem: All resources in single state | || | Impact: Slow plans, blast radius of errors too large | || | Fix: Split by service/layer, use data sources/outputs | || +----------------------------------------------------------+ || || ❌ Mistake 4: `terraform apply -auto-approve` in Production || +----------------------------------------------------------+ || | Problem: No human review before production changes | || | Impact: Accidental resource deletion or misconfiguration| || | Fix: Always review plan output, use -auto-approve only | || | in CI/CD with separate plan+apply stages | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+41.15 Interview Questions
Section titled “41.15 Interview Questions”Conceptual Questions
Section titled “Conceptual Questions”-
Q: Explain Terraform state and why it’s important.
- A: State is Terraform’s record of what infrastructure exists. It maps configuration to real-world resources, tracks metadata, and enables plan/apply to determine what needs to change. Without state, Terraform can’t know what already exists. Store remotely in S3 with DynamoDB locking for teams. Never commit to Git (contains secrets). Enable versioning on S3 bucket for recovery.
-
Q: How do you handle secrets in Terraform?
- A: (1) Never hardcode secrets in
.tffiles, (2) Usesensitive = trueon variables, (3) Reference Secrets Manager/SSM Parameter Store via data sources, (4) Use environment variables (TF_VAR_*), (5) Terraform Cloud/Enterprise has encrypted variable storage, (6) Enable S3 state encryption with KMS. State file itself contains secrets, so protect access.
- A: (1) Never hardcode secrets in
Scenario-Based Questions
Section titled “Scenario-Based Questions”- Q: How would you migrate existing AWS infrastructure to Terraform?
- A: (1) Use
terraform importto bring resources into state, (2) Write corresponding HCL configuration, (3) Runterraform planto verify no changes needed (should show “No changes”), (4) Use tools liketerraformerfor bulk import, (5) Start with non-critical resources, (6) Test in a separate workspace first, (7) Gradually expand coverage.
- A: (1) Use
41.16 Exam Tips
Section titled “41.16 Exam Tips”
Key Exam Points+------------------------------------------------------------------+| || 1. Terraform uses HCL (HashiCorp Configuration Language) || || 2. State tracks infrastructure and must be protected || || 3. Use S3 + DynamoDB for remote state with locking || || 4. terraform plan shows changes before apply || || 5. terraform import brings existing resources under management || || 6. Modules enable code reuse and organization || || 7. Workspaces separate environments || || 8. Providers connect Terraform to cloud platforms || || 9. Use terraform destroy to remove all resources || || 10. State file is sensitive - never commit to Git || |+------------------------------------------------------------------+41.17 Summary
Section titled “41.17 Summary” Chapter 41 Summary+------------------------------------------------------------------+| || Terraform Core Concepts || +------------------------------------------------------------+ || | - HCL: Declarative configuration language | || | - State: Infrastructure tracking | || | - Providers: Cloud integrations | || | - Modules: Reusable configurations | || +------------------------------------------------------------+ || || Key Commands || +------------------------------------------------------------+ || | - init: Initialize configuration | || | - plan: Preview changes | || | - apply: Execute changes | || | - destroy: Remove infrastructure | || +------------------------------------------------------------+ || || Best Practices || +------------------------------------------------------------+ || | - Remote state with locking | || | - Use modules | || | - Version control | || | - Use workspaces | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Next Chapter: Chapter 42: Packer - Machine Image Building