AWS Lambda & Serverless
Chapter 8: AWS Lambda - Serverless Computing
Section titled “Chapter 8: AWS Lambda - Serverless Computing”Building Event-Driven Serverless Applications
Section titled “Building Event-Driven Serverless Applications”8.1 Overview
Section titled “8.1 Overview”AWS Lambda is a serverless compute service that runs your code in response to events and automatically manages the underlying compute resources.
Lambda Core Concepts+------------------------------------------------------------------+| || +------------------------+ || | AWS Lambda | || +------------------------+ || | || +-----------+-----------+-----------+-----------+ || | | | | | || v v v v v || +-------+ +-------+ +-------+ +-------+ +-------+ || |Triggers| |Function| |Runtime| |Layers | |Config | || | | | Code | | | | | | | || +-------+ +-------+ +-------+ +-------+ +-------+ || || Triggers: Event sources that invoke Lambda || Function Code: Your application logic || Runtime: Execution environment (Node.js, Python, etc.) || Layers: Shared code and dependencies || Config: Memory, timeout, environment variables || |+------------------------------------------------------------------+8.2 Lambda Execution Model
Section titled “8.2 Lambda Execution Model”Execution Environment
Section titled “Execution Environment” Lambda Execution Environment+------------------------------------------------------------------+| || +----------------------------------------------------------+ || | Execution Environment | || | | || | +------------------+ +------------------------+ | || | | Runtime API | | Your Function Code | | || | | | | | | || | | - Node.js | | exports.handler = | | || | | - Python | | async (event) => { | | || | | - Java | | // Your code | | || | | - Go | | } | | || | | - .NET | | | | || | | - Ruby | +------------------------+ | || | | - Custom | | || | +------------------+ | || | | || | +------------------+ +------------------------+ | || | | /tmp Storage | | Environment Vars | | || | | (512 MB - 10GB)| | (Key-Value pairs) | | || | +------------------+ +------------------------+ | || | | || +----------------------------------------------------------+ || || Lifecycle: || 1. INIT phase - Load function, run initialization code || 2. INVOKE phase - Process events || 3. SHUTDOWN phase - Clean up (on environment shutdown) || |+------------------------------------------------------------------+Cold vs Warm Starts
Section titled “Cold vs Warm Starts” Cold Start vs Warm Start+------------------------------------------------------------------+| || Cold Start || +----------------------------------------------------------+ || | | || | Request arrives | || | | | || | v | || | +----------+ +----------+ +----------+ | || | | Download | --> | Initialize| --> | Execute | | || | | Code | | Runtime | | Function | | || | +----------+ +----------+ +----------+ | || | | | | | || | v v v | || | ~100ms ~200ms ~50ms | || | | || | Total: ~350ms (can be higher for Java/.NET) | || +----------------------------------------------------------+ || || Warm Start || +----------------------------------------------------------+ || | | || | Request arrives | || | | | || | v | || | +----------+ +----------+ | || | | Reuse | --> | Execute | | || | | Container| | Function | | || | +----------+ +----------+ | || | | || | Total: ~50ms (much faster!) | || +----------------------------------------------------------+ || || Reducing Cold Starts: || +----------------------------------------------------------+ || | - Use Provisioned Concurrency | || | - Keep functions small | || | - Use interpreted languages (Node.js, Python) | || | - Minimize dependencies | || | - Use SnapStart (Java) | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+8.3 Lambda Triggers
Section titled “8.3 Lambda Triggers”Event Source Mapping
Section titled “Event Source Mapping” Lambda Trigger Sources+------------------------------------------------------------------+| || Push Model (Synchronous) || +----------------------------------------------------------+ || | | || | API Gateway User Services | || | +----------+ +----------+ | || | | HTTP | | Custom | | || | | Request | ------> | SDK | ------> Lambda | || | +----------+ +----------+ | || | | || | - Caller waits for response | || | - Error handling by caller | || +----------------------------------------------------------+ || || Push Model (Asynchronous) || +----------------------------------------------------------+ || | | || | S3 SNS EventBridge | || | +----------+ +----------+ +----------+ | || | | Object | | Message | | Schedule | | || | | Upload | -----> | Publish | -----> | Rule | --+--> Lambda| | +----------+ +----------+ +----------+ | || | | || | - Lambda retries on failure (0-2 times) | || | - Can configure DLQ | || +----------------------------------------------------------+ || || Poll-Based Model || +----------------------------------------------------------+ || | | || | Kinesis DynamoDB SQS | || | +----------+ +----------+ +----------+ | || | | Stream | | Stream | | Queue | | || | +----------+ +----------+ +----------+ | || | | | | | || | +---------+---------+---------+---------+ | || | | | | || | v v | || | Lambda (polls for records) | || | | || | - Lambda manages polling | || | - Batch size configurable | || | - Checkpointing handled | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+Trigger Configuration Examples
Section titled “Trigger Configuration Examples” Common Trigger Configurations+------------------------------------------------------------------+| || API Gateway Trigger || +----------------------------------------------------------+ || | | || | Client -> API Gateway -> Lambda | || | | || | Event Structure: | || | { | || | "httpMethod": "POST", | || | "path": "/users", | || | "headers": { "Content-Type": "application/json" }, | || | "body": "{\"name\": \"John\"}" | || | } | || +----------------------------------------------------------+ || || S3 Trigger || +----------------------------------------------------------+ || | | || | S3 Upload -> Lambda | || | | || | Event Structure: | || | { | || | "Records": [{ | || | "s3": { | || | "bucket": { "name": "my-bucket" }, | || | "object": { "key": "uploads/file.pdf" } | || | } | || | }] | || | } | || +----------------------------------------------------------+ || || DynamoDB Stream Trigger || +----------------------------------------------------------+ || | | || | DynamoDB -> Stream -> Lambda | || | | || | Event Structure: | || | { | || | "Records": [{ | || | "eventName": "INSERT", | || | "dynamodb": { | || | "Keys": { "id": "123" }, | || | "NewImage": { "name": "John" } | || | } | || | }] | || | } | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+8.4 Lambda Function Configuration
Section titled “8.4 Lambda Function Configuration”Memory and Timeout
Section titled “Memory and Timeout” Lambda Configuration Options+------------------------------------------------------------------+| || Memory Configuration || +----------------------------------------------------------+ || | | || | Memory Range: 128 MB - 10,240 MB (in 1 MB increments) | || | | || | Memory vCPU Network Performance | || | +--------+-------+-------------------+ | || | | 128 MB | ~1 | Very Low | | || | | 512 MB | ~1 | Low | | || | | 1024 MB| ~1 | Moderate | | || | | 1769 MB| ~1 | Up to 1 Gbps | | || | | 2048 MB| ~2 | Up to 1.5 Gbps | | || | | 4096 MB| ~3 | Up to 2.5 Gbps | | || | | 10240 MB| ~6 | Up to 10 Gbps | | || | +--------+-------+-------------------+ | || | | || | Note: More memory = more CPU power | || +----------------------------------------------------------+ || || Timeout Configuration || +----------------------------------------------------------+ || | | || | Range: 1 second - 15 minutes | || | | || | Best Practice: | || | - Set timeout slightly above expected execution time | || | - Consider retry logic for longer operations | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+Environment Variables
Section titled “Environment Variables” Lambda Environment Variables+------------------------------------------------------------------+| || Configuration: || +----------------------------------------------------------+ || | | || | Key Value | || | +--------------------+--------------------+ | || | | DB_TABLE | users | | || | | S3_BUCKET | my-data-bucket | | || | | LOG_LEVEL | INFO | | || | | API_ENDPOINT | https://api.example| | || | +--------------------+--------------------+ | || | | || | Access in code: | || | process.env.DB_TABLE // Node.js | || | os.environ['DB_TABLE'] // Python | || | System.getenv("DB_TABLE") // Java | || +----------------------------------------------------------+ || || Reserved Environment Variables: || +----------------------------------------------------------+ || | AWS_REGION - Region (e.g., us-east-1) | || | AWS_LAMBDA_FUNCTION_NAME - Function name | || | AWS_LAMBDA_FUNCTION_MEMORY_SIZE - Memory in MB | || | AWS_LAMBDA_FUNCTION_VERSION - Version | || | AWS_EXECUTION_ENV - Runtime identifier | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+8.5 Lambda Function Code
Section titled “8.5 Lambda Function Code”Handler Patterns
Section titled “Handler Patterns”# Python Handler Exampleimport jsonimport boto3
# Initialize outside handler for connection reusedynamodb = boto3.resource('dynamodb')table = dynamodb.Table('users')
def lambda_handler(event, context): """ Main handler function
Args: event: Event data from trigger context: Runtime information (request_id, function_name, etc.)
Returns: Response object """ try: # Parse input user_id = event['pathParameters']['id']
# Business logic response = table.get_item(Key={'id': user_id})
if 'Item' not in response: return { 'statusCode': 404, 'body': json.dumps({'error': 'User not found'}) }
# Return response return { 'statusCode': 200, 'headers': { 'Content-Type': 'application/json' }, 'body': json.dumps(response['Item']) }
except Exception as e: # Error handling print(f"Error: {str(e)}") return { 'statusCode': 500, 'body': json.dumps({'error': str(e)}) }// Node.js Handler Exampleconst AWS = require('aws-sdk');const dynamodb = new AWS.DynamoDB.DocumentClient();
// Initialize outside handler for connection reuseconst USERS_TABLE = process.env.USERS_TABLE;
exports.handler = async (event) => { try { // Parse input const userId = event.pathParameters.id;
// Business logic const params = { TableName: USERS_TABLE, Key: { id: userId } };
const result = await dynamodb.get(params).promise();
if (!result.Item) { return { statusCode: 404, body: JSON.stringify({ error: 'User not found' }) }; }
// Return response return { statusCode: 200, headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(result.Item) };
} catch (error) { // Error handling console.error('Error:', error); return { statusCode: 500, body: JSON.stringify({ error: error.message }) }; }};8.6 Lambda Layers
Section titled “8.6 Lambda Layers”Layer Architecture
Section titled “Layer Architecture” Lambda Layers Structure+------------------------------------------------------------------+| || Function with Layers || +----------------------------------------------------------+ || | | || | /opt (Layers mount point) | || | +--------------------------------------------------+ | || | | Layer 1: /opt/nodejs/node_modules/ | | || | | (Shared libraries) | | || | +--------------------------------------------------+ | || | +--------------------------------------------------+ | || | | Layer 2: /opt/python/lib/python3.9/site-packages/ | | || | | (Python dependencies) | | || | +--------------------------------------------------+ | || | +--------------------------------------------------+ | || | | Layer 3: /opt/bin/ | | || | | (Custom binaries) | | || | +--------------------------------------------------+ | || | | || | Function Code: /var/task/ | || | | || +----------------------------------------------------------+ || || Layer Contents: || +----------------------------------------------------------+ || | - Runtime-specific libraries | || | - Custom runtimes | || | - Shared code across functions | || | - Configuration files | || | - Native libraries | || +----------------------------------------------------------+ || || Benefits: || +----------------------------------------------------------+ || | - Reduce deployment package size | || | - Share code across functions | || | - Separate dependencies from code | || | - Version control for dependencies | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+8.7 Lambda Deployment
Section titled “8.7 Lambda Deployment”Deployment Package
Section titled “Deployment Package” Lambda Deployment Options+------------------------------------------------------------------+| || 1. Zip File Deployment || +----------------------------------------------------------+ || | | || | Size Limits: | || | - Direct upload: 50 MB (zipped) | || | - S3 upload: 250 MB (zipped) | || | - Unzipped: 250 MB | || | | || | Structure: | || | function.zip | || | +-- lambda_function.py | || | +-- requirements.txt | || | +-- utils/ | || | +-- helpers.py | || +----------------------------------------------------------+ || || 2. Container Image Deployment || +----------------------------------------------------------+ || | | || | Size Limit: 10 GB | || | | || | Dockerfile: | || | FROM public.ecr.aws/lambda/python:3.9 | || | | || | COPY requirements.txt . | || | RUN pip install -r requirements.txt | || | | || | COPY lambda_function.py . | || | CMD ["lambda_function.handler"] | || | | || | Benefits: | || | - Larger dependencies | || | - Custom runtimes | || | - Consistent local testing | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+Deployment with AWS CLI
Section titled “Deployment with AWS CLI”# Create deployment packagezip -r function.zip lambda_function.py
# Create functionaws lambda create-function \ --function-name my-function \ --runtime python3.9 \ --role arn:aws:iam::123456789012:role/lambda-role \ --handler lambda_function.handler \ --zip-file fileb://function.zip
# Update function codeaws lambda update-function-code \ --function-name my-function \ --zip-file fileb://function.zip
# Update function configurationaws lambda update-function-configuration \ --function-name my-function \ --timeout 30 \ --memory-size 512 \ --environment Variables={DB_TABLE=users,LOG_LEVEL=INFO}
# Publish versionaws lambda publish-version \ --function-name my-function \ --description "Production version"
# Create aliasaws lambda create-alias \ --function-name my-function \ --name production \ --function-version 18.8 Lambda Best Practices
Section titled “8.8 Lambda Best Practices”Performance Best Practices
Section titled “Performance Best Practices” Lambda Performance Optimization+------------------------------------------------------------------+| || 1. Minimize Cold Starts || +----------------------------------------------------------+ || | - Use Provisioned Concurrency for critical functions | || | - Keep deployment package small | || | - Initialize outside handler | || | - Use SnapStart for Java (if available in region) | || +----------------------------------------------------------+ || || 2. Optimize Memory || +----------------------------------------------------------+ || | - Test different memory settings | || | - Higher memory = more CPU | || | - Use CloudWatch Lambda Insights | || | - Balance cost vs performance | || +----------------------------------------------------------+ || || 3. Connection Reuse || +----------------------------------------------------------+ || | - Initialize SDK clients outside handler | || | - Use connection pooling | || | - Keep connections alive | || +----------------------------------------------------------+ || || 4. Efficient Code || +----------------------------------------------------------+ || | - Avoid heavy initialization | || | - Use lazy loading | || | - Minimize dependencies | || | - Use Layers for shared code | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+Security Best Practices
Section titled “Security Best Practices” Lambda Security Checklist+------------------------------------------------------------------+| || 1. IAM Roles || +----------------------------------------------------------+ || | [ ] Use least privilege permissions | || | [ ] Separate roles for different functions | || | [ ] Avoid using '*' in resource policies | || +----------------------------------------------------------+ || || 2. Environment Variables || +----------------------------------------------------------+ || | [ ] Use KMS encryption for secrets | || | [ ] Use AWS Secrets Manager for sensitive data | || | [ ] Don't log sensitive information | || +----------------------------------------------------------+ || || 3. VPC Configuration || +----------------------------------------------------------+ || | [ ] Use VPC for RDS/ElastiCache access | || | [ ] Use VPC endpoints for AWS services | || | [ ] Configure security groups properly | || +----------------------------------------------------------+ || || 4. Code Security || +----------------------------------------------------------+ || | [ ] Validate input data | || | [ ] Handle errors properly | || | [ ] Use parameterized queries | || | [ ] Keep dependencies updated | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+8.9 Lambda Limits
Section titled “8.9 Lambda Limits”| Resource | Limit |
|---|---|
| Memory | 128 MB - 10,240 MB |
| Timeout | 15 minutes |
| Deployment package (zip) | 50 MB (direct), 250 MB (S3) |
| Container image | 10 GB |
| Environment variables | 4 KB |
| Layers | 5 layers, 250 MB total |
| Concurrent executions | 1,000 (default, adjustable) |
| /tmp storage | 512 MB - 10,240 MB |
8.10 Why This Matters in DevOps/SRE
Section titled “8.10 Why This Matters in DevOps/SRE”Lambda is a game-changer for DevOps automation. From event-driven infrastructure tasks to lightweight API backends, Lambda eliminates server management overhead.
Lambda in DevOps/SRE+------------------------------------------------------------------+| || Common DevOps Lambda Use Cases: || || 1. Automation Glue || +----------------------------------------------------------+ || | - Rotate IAM keys, clean orphaned resources | || | - Auto-tag resources on creation (via CloudTrail) | || | - Slack/PagerDuty alerts from CloudWatch alarms | || +----------------------------------------------------------+ || || 2. CI/CD Pipeline Steps || +----------------------------------------------------------+ || | - Pre/post deployment hooks | || | - Smoke tests after deployment | || | - Custom deployment notifications | || +----------------------------------------------------------+ || || 3. Infrastructure Maintenance || +----------------------------------------------------------+ || | - EBS snapshot management | || | - Log rotation and cleanup | || | - Scheduled health checks | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+8.11 Linux Systems Perspective
Section titled “8.11 Linux Systems Perspective”Lambda Development from Arch Linux
Section titled “Lambda Development from Arch Linux”# Install Lambda development tools on Arch Linuxsudo pacman -S python python-pip dockeryay -S aws-sam-cli # Serverless Application Model
# Local Lambda development workflowmkdir -p ~/lambda-project && cd ~/lambda-project
# Create SAM templatecat > template.yaml << 'EOF'AWSTemplateFormatVersion: '2010-09-09'Transform: AWS::Serverless-2016-10-31
Resources: CleanupFunction: Type: AWS::Serverless::Function Properties: Handler: handler.lambda_handler Runtime: python3.11 Timeout: 300 MemorySize: 256 Events: ScheduleEvent: Type: Schedule Properties: Schedule: rate(1 day) Policies: - Version: '2012-10-17' Statement: - Effect: Allow Action: - ec2:DescribeVolumes - ec2:DeleteVolume Resource: '*'EOF
# Test locally with SAMsam local invoke CleanupFunction --event events/test.jsonsam local start-api # Run API Gateway locally
# Deploy Lambda function via CLIzip -j function.zip handler.pyaws lambda update-function-code \ --function-name cleanup-orphaned-ebs \ --zip-file fileb://function.zip
# View Lambda logs in real-timeaws logs tail /aws/lambda/my-function --follow --format short
# Invoke and capture outputaws lambda invoke \ --function-name my-function \ --payload '{"key": "value"}' \ --cli-binary-format raw-in-base64-out \ /dev/stdout 2>/dev/null | jq .8.12 Troubleshooting Guide
Section titled “8.12 Troubleshooting Guide”| Issue | Cause | Solution |
|---|---|---|
| High cold start latency | Large deployment package | Minimize dependencies, use layers |
| Function timeout (15 min) | Processing too slow | Optimize code, increase memory (more CPU) |
Task timed out after X seconds | Timeout too low | Increase timeout in configuration |
| Cannot reach RDS/ElastiCache | Lambda not in VPC | Configure VPC, subnets, and security groups |
| Throttling (429 errors) | Concurrent execution limit hit | Request limit increase or use reserved concurrency |
| Permission denied errors | Execution role missing perms | Update IAM role with required permissions |
# Debug Lambda issues# Check recent invocations and errorsaws logs filter-log-events \ --log-group-name /aws/lambda/my-function \ --filter-pattern "ERROR" \ --start-time $(date -d '1 hour ago' +%s000) \ --query 'events[*].[timestamp,message]' \ --output text
# Check concurrencyaws lambda get-function-concurrency \ --function-name my-function
# Check function configurationaws lambda get-function-configuration \ --function-name my-function \ --query '{Memory:MemorySize,Timeout:Timeout,Runtime:Runtime,LastModified:LastModified}'8.13 Common Mistakes & Anti-Patterns
Section titled “8.13 Common Mistakes & Anti-Patterns” Lambda Anti-Patterns+------------------------------------------------------------------+| || ❌ Mistake 1: Monolithic Lambda Functions || +----------------------------------------------------------+ || | Problem: One huge function doing everything | || | Impact: Slow cold starts, hard to debug | || | Fix: Single responsibility, one function per task | || +----------------------------------------------------------+ || || ❌ Mistake 2: Initializing SDK Inside Handler || +----------------------------------------------------------+ || | Problem: Creating boto3 clients on every invocation | || | Impact: Wasted time on warm starts | || | Fix: Initialize outside handler for connection reuse | || +----------------------------------------------------------+ || || ❌ Mistake 3: Not Setting Reserved Concurrency || +----------------------------------------------------------+ || | Problem: One function consuming all account concurrency | || | Impact: Other functions get throttled | || | Fix: Set reserved concurrency for critical functions | || +----------------------------------------------------------+ || || ❌ Mistake 4: Lambda for Long-Running Tasks || +----------------------------------------------------------+ || | Problem: Trying to run 30+ minute jobs in Lambda | || | Impact: Timeout failures, incomplete processing | || | Fix: Use Step Functions, ECS/Fargate, or SQS batches | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+8.14 Interview Questions
Section titled “8.14 Interview Questions”Conceptual Questions
Section titled “Conceptual Questions”-
Q: Explain cold starts and how to mitigate them.
- A: Cold starts occur when Lambda creates a new execution environment — downloading code, initializing runtime, running init code. Mitigation: (1) Provisioned Concurrency pre-warms containers, (2) Keep packages small, (3) Initialize SDK clients outside handler, (4) Use interpreted languages over compiled ones for faster startup, (5) SnapStart for Java.
-
Q: What’s the difference between synchronous and asynchronous invocation?
- A: Synchronous: caller waits for response (API Gateway, SDK invoke). Asynchronous: caller gets immediate ACK, Lambda retries on failure (S3, SNS, EventBridge). Async invocations support DLQ/destinations for failed events. Use sync for user-facing APIs, async for background processing.
Scenario-Based Questions
Section titled “Scenario-Based Questions”- Q: Design a serverless image processing pipeline.
- A: S3 upload triggers Lambda → Lambda validates image, creates thumbnail via Pillow/Sharp → stores thumbnail in S3 output bucket → writes metadata to DynamoDB → sends notification via SNS. For large images, use Step Functions to orchestrate resize/watermark/optimize steps. Use SQS between S3 and Lambda for spike buffering.
8.15 Exam Tips
Section titled “8.15 Exam Tips”
- Execution Model: Cold starts vs warm starts, provisioned concurrency
- Triggers: Push vs poll model, synchronous vs asynchronous
- Memory: More memory = more CPU power
- Timeout: Maximum 15 minutes
- Layers: Share code and dependencies across functions
- VPC: Required for accessing RDS, ElastiCache in VPC
- Reserved Concurrency: Guarantees capacity for critical functions
- Event Sources: Know which use push vs poll model
Next Chapter
Section titled “Next Chapter”Chapter 9: AWS Elastic Container Services (ECS/EKS)
Last Updated: March 2026