Pipeline
Chapter 32: AWS CodePipeline & CI/CD Best Practices
Section titled “Chapter 32: AWS CodePipeline & CI/CD Best Practices”Continuous Delivery Orchestration
Section titled “Continuous Delivery Orchestration”32.1 Overview
Section titled “32.1 Overview”AWS CodePipeline is a fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates.
AWS CodePipeline Overview+------------------------------------------------------------------+| || +------------------------+ || | AWS CodePipeline | || +------------------------+ || | || +---------------------+---------------------+ || | | | || v v v || +----------+ +----------+ +----------+ || | Stages | | Actions | | Transitions || | | | | | | || | - Source | | - Source | | - Manual | || | - Build | | - Build | | - Auto | || | - Deploy | | - Deploy | | - Conditions || +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+Pipeline Structure
Section titled “Pipeline Structure” CodePipeline Structure+------------------------------------------------------------------+| || Pipeline || +------------------------------------------------------------+ || | | || | Stage 1: Source Stage 2: Build Stage 3: Deploy || | +----------------+ +----------------+ +----------------+ || | | Action 1 | --> | Action 1 | --> | Action 1 | || | | (CodeCommit) | | (CodeBuild) | | (CodeDeploy) | || | +----------------+ +----------------+ +----------------+ || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+32.2 Pipeline Components
Section titled “32.2 Pipeline Components”Stages
Section titled “Stages” CodePipeline Stages+------------------------------------------------------------------+| || Source Stage || +------------------------------------------------------------+ || | | || | Supported Sources: | || | +------------------------------------------------------+ | || | | - AWS CodeCommit | | || | | - GitHub | | || | | - Bitbucket | | || | | - Amazon S3 | | || | | - Amazon ECR | | || | +------------------------------------------------------+ | || | | || +------------------------------------------------------------+ || || Build Stage || +------------------------------------------------------------+ || | | || | Supported Build Providers: | || | +------------------------------------------------------+ | || | | - AWS CodeBuild | | || | | - Jenkins | | || | | - TeamCity | | || | | - CloudBees | | || | +------------------------------------------------------+ | || | | || +------------------------------------------------------------+ || || Deploy Stage || +------------------------------------------------------------+ || | | || | Supported Deploy Providers: | || | +------------------------------------------------------+ | || | | - AWS CodeDeploy (EC2, Lambda, ECS) | | || | | - AWS CloudFormation | | || | | - AWS Elastic Beanstalk | | || | | - AWS ECS | | || | | - AWS S3 | | || | | - AWS AppConfig | | || | +------------------------------------------------------+ | || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Action Types
Section titled “Action Types” CodePipeline Action Types+------------------------------------------------------------------+| || Source Actions || +------------------------------------------------------------+ || | - Checkout code from repository | || | - Trigger pipeline on changes | || | - Output: source artifacts | || +------------------------------------------------------------+ || || Build Actions || +------------------------------------------------------------+ || | - Compile source code | || | - Run tests | || | - Output: build artifacts | || +------------------------------------------------------------+ || || Deploy Actions || +------------------------------------------------------------+ || | - Deploy to target environment | || | - Update infrastructure | || | - Output: deployment status | || +------------------------------------------------------------+ || || Approval Actions || +------------------------------------------------------------+ || | - Manual approval gate | || | - Require human review | || | - Output: approval status | || +------------------------------------------------------------+ || || Test Actions || +------------------------------------------------------------+ || | - Run automated tests | || | - Integration tests | || | - Output: test results | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+32.3 Pipeline Configuration
Section titled “32.3 Pipeline Configuration”Pipeline Definition (JSON)
Section titled “Pipeline Definition (JSON)” CodePipeline Definition+------------------------------------------------------------------+| || { || "pipeline": { || "name": "MyAppPipeline", || "roleArn": "arn:aws:iam::123456789012:role/CodePipelineRole", || "artifactStore": { || "type": "S3", || "location": "my-pipeline-artifacts" || }, || "stages": [ || { || "name": "Source", || "actions": [ || { || "name": "SourceAction", || "actionTypeId": { || "category": "Source", || "owner": "AWS", || "provider": "CodeCommit", || "version": "1" || }, || "outputArtifacts": [ || { "name": "SourceOutput" } || ], || "configuration": { || "RepositoryName": "my-repo", || "BranchName": "main" || } || } || ] || }, || { || "name": "Build", || "actions": [ || { || "name": "BuildAction", || "actionTypeId": { || "category": "Build", || "owner": "AWS", || "provider": "CodeBuild", || "version": "1" || }, || "inputArtifacts": [ || { "name": "SourceOutput" } || ], || "outputArtifacts": [ || { "name": "BuildOutput" } || ], || "configuration": { || "ProjectName": "my-build-project" || } || } || ] || }, || { || "name": "Deploy", || "actions": [ || { || "name": "DeployAction", || "actionTypeId": { || "category": "Deploy", || "owner": "AWS", || "provider": "CodeDeploy", || "version": "1" || }, || "inputArtifacts": [ || { "name": "BuildOutput" } || ], || "configuration": { || "ApplicationName": "my-app", || "DeploymentGroupName": "my-deployment-group" || } || } || ] || } || ] || } || } || |+------------------------------------------------------------------+32.4 Advanced Pipeline Patterns
Section titled “32.4 Advanced Pipeline Patterns”Multi-Stage Pipeline
Section titled “Multi-Stage Pipeline” Multi-Stage Pipeline Pattern+------------------------------------------------------------------+| || +----------+ +----------+ +----------+ +----------+ || | Source | -->| Build | -->| Test | -->| Approval | || | | | | | | | | || | CodeCommit| | CodeBuild| | CodeBuild| | Manual | || +----------+ +----------+ +----------+ +----------+ || | || v || +----------+ +----------+ +----------+ || | Prod | <--| Staging | <--| Deploy | || | Deploy | | Deploy | | Dev | || | | | | | | || | CodeDeploy| | CodeDeploy| | CodeDeploy| || +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+Blue/Green Deployment Pipeline
Section titled “Blue/Green Deployment Pipeline” Blue/Green Deployment Pipeline+------------------------------------------------------------------+| || +----------+ +----------+ +----------+ +----------+ || | Source | -->| Build | -->| Test | -->| Create | || | | | | | | | Green | || +----------+ +----------+ +----------+ +----------+ || | || v || +----------+ +----------+ +----------+ || | Terminate| <--| Shift | <--| Validate | || | Blue | | Traffic | | Green | || +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+Cross-Account Pipeline
Section titled “Cross-Account Pipeline” Cross-Account Pipeline+------------------------------------------------------------------+| || Dev Account Prod Account || +------------------+ +------------------+ || | | | | || | +----------+ | | +----------+ | || | | Source | | | | Deploy | | || | | (CodeCommit) | | | (CodeDeploy) | || | +----------+ | | +----------+ | || | | | | ^ | || | v | | | | || | +----------+ | | | | || | | Build | | | | | || | | (CodeBuild) | | | | || | +----------+ | | | | || | | | | | | || | +---------+------------>+--------+ | || | | | | || +------------------+ +------------------+ || |+------------------------------------------------------------------+32.5 CodePipeline CLI Commands
Section titled “32.5 CodePipeline CLI Commands”# Create pipelineaws codepipeline create-pipeline \ --cli-input-json file://pipeline-definition.json
# Get pipelineaws codepipeline get-pipeline \ --name my-pipeline
# List pipelinesaws codepipeline list-pipelines
# Update pipelineaws codepipeline update-pipeline \ --cli-input-json file://pipeline-definition.json
# Start pipeline executionaws codepipeline start-pipeline-execution \ --name my-pipeline
# List pipeline executionsaws codepipeline list-pipeline-executions \ --pipeline-name my-pipeline
# Get pipeline executionaws codepipeline get-pipeline-execution \ --pipeline-name my-pipeline \ --pipeline-execution-id execution-id
# Retry stage executionaws codepipeline retry-stage-execution \ --pipeline-name my-pipeline \ --stage-name Build \ --pipeline-execution-id execution-id \ --retry-mode FAILED_ACTIONS
# Stop pipeline executionaws codepipeline stop-pipeline-execution \ --pipeline-name my-pipeline \ --pipeline-execution-id execution-id
# Put approval resultaws codepipeline put-approval-result \ --pipeline-name my-pipeline \ --stage-name Approval \ --action-name ApprovalAction \ --result summary="Approved",status=Approved \ --token approval-token
# Delete pipelineaws codepipeline delete-pipeline \ --name my-pipeline32.6 CI/CD Best Practices
Section titled “32.6 CI/CD Best Practices”Pipeline Design Best Practices
Section titled “Pipeline Design Best Practices” Pipeline Design Best Practices+------------------------------------------------------------------+| || 1. Keep pipelines simple and focused || +------------------------------------------------------------+ || | - Single responsibility per pipeline | || | - Separate pipelines for different environments | || +------------------------------------------------------------+ || || 2. Implement proper artifact management || +------------------------------------------------------------+ || | - Use versioned artifacts | || | - Store artifacts in S3 with lifecycle policies | || +------------------------------------------------------------+ || || 3. Use manual approvals for production || +------------------------------------------------------------+ || | - Require human approval before production deployment | || | - Include relevant stakeholders | || +------------------------------------------------------------+ || || 4. Implement automated testing || +------------------------------------------------------------+ || | - Unit tests in build stage | || | - Integration tests in test stage | || | - Security scanning | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Security Best Practices
Section titled “Security Best Practices” CI/CD Security Best Practices+------------------------------------------------------------------+| || 1. Use least privilege IAM roles || +------------------------------------------------------------+ || | - Separate roles for each service | || | - Use resource-level permissions | || +------------------------------------------------------------+ || || 2. Secure secrets management || +------------------------------------------------------------+ || | - Use Secrets Manager or Parameter Store | || | - Never store secrets in code | || | - Rotate secrets regularly | || +------------------------------------------------------------+ || || 3. Enable encryption || +------------------------------------------------------------+ || | - Encrypt artifacts at rest | || | - Use KMS for encryption keys | || +------------------------------------------------------------+ || || 4. Implement audit logging || +------------------------------------------------------------+ || | - Enable CloudTrail for all CI/CD services | || | - Monitor pipeline execution logs | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Performance Best Practices
Section titled “Performance Best Practices” CI/CD Performance Best Practices+------------------------------------------------------------------+| || 1. Optimize build time || +------------------------------------------------------------+ || | - Use caching for dependencies | || | - Parallelize tests | || | - Use appropriate compute types | || +------------------------------------------------------------+ || || 2. Minimize artifact size || +------------------------------------------------------------+ || | - Only include necessary files | || | - Use .gitignore and exclude patterns | || +------------------------------------------------------------+ || || 3. Use incremental deployments || +------------------------------------------------------------+ || | - Deploy only changed components | || | - Use rolling updates | || +------------------------------------------------------------+ || || 4. Implement proper monitoring || +------------------------------------------------------------+ || | - Set up CloudWatch alarms | || | - Monitor pipeline execution duration | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+32.7 Integration with Other Services
Section titled “32.7 Integration with Other Services”EventBridge Integration
Section titled “EventBridge Integration” EventBridge Integration+------------------------------------------------------------------+| || CodeCommit Event --> EventBridge --> CodePipeline || || Event Pattern: || +------------------------------------------------------------+ || | { | || | "source": ["aws.codecommit"], | || | "detail-type": ["CodeCommit Repository State Change"], | || | "detail": { | || | "event": ["push"], | || | "repositoryName": ["my-repo"], | || | "referenceName": ["main"] | || | } | || | } | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+SNS Notifications
Section titled “SNS Notifications” SNS Notification Integration+------------------------------------------------------------------+| || Pipeline Events --> SNS Topic --> Subscribers || || Notification Types: || +------------------------------------------------------------+ || | - Pipeline execution started | || | - Pipeline execution succeeded | || | - Pipeline execution failed | || | - Stage execution started/succeeded/failed | || | - Action execution started/succeeded/failed | || | - Manual approval needed | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+32.8 Troubleshooting
Section titled “32.8 Troubleshooting”Common Issues
Section titled “Common Issues” Common CodePipeline Issues+------------------------------------------------------------------+| || Issue 1: Pipeline stuck in progress || +------------------------------------------------------------+ || | Cause: Action timeout or resource unavailable | || | Solution: Check action logs, increase timeout | || +------------------------------------------------------------+ || || Issue 2: Artifact not found || +------------------------------------------------------------+ || | Cause: Incorrect artifact name or missing output | || | Solution: Verify artifact names match between stages | || +------------------------------------------------------------+ || || Issue 3: Permission denied || +------------------------------------------------------------+ || | Cause: Missing IAM permissions | || | Solution: Check service role permissions | || +------------------------------------------------------------+ || || Issue 4: Source change not triggering pipeline || +------------------------------------------------------------+ || | Cause: Webhook not configured or disabled | || | Solution: Check CloudWatch Events rule for source | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+32.9 Exam Tips
Section titled “32.9 Exam Tips”
Key Exam Points+------------------------------------------------------------------+| || 1. CodePipeline orchestrates the entire CI/CD workflow || || 2. Pipelines consist of stages, actions, and transitions || || 3. Each stage can have multiple actions (sequential/parallel) || || 4. Artifacts are passed between stages via S3 || || 5. Manual approval actions require human intervention || || 6. EventBridge triggers pipelines on source changes || || 7. Cross-account pipelines use KMS encryption || || 8. CodePipeline supports third-party integrations || || 9. Use retry-stage-execution to retry failed actions || || 10. Pipeline execution can be stopped mid-execution || |+------------------------------------------------------------------+32.10 Summary
Section titled “32.10 Summary” Chapter 32 Summary+------------------------------------------------------------------+| || AWS CodePipeline || +------------------------------------------------------------+ || | - Fully managed continuous delivery service | || | - Visual workflow for release process | || | - Integrates with AWS and third-party services | || +------------------------------------------------------------+ || || Pipeline Components || +------------------------------------------------------------+ || | - Stages: Source, Build, Test, Deploy, Approval | || | - Actions: Individual operations within stages | || | - Artifacts: Files passed between stages | || | - Transitions: Movement between stages | || +------------------------------------------------------------+ || || Best Practices || +------------------------------------------------------------+ || | - Implement automated testing | || | - Use manual approvals for production | || | - Secure secrets properly | || | - Monitor pipeline performance | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Previous Chapter: Chapter 31: AWS CodeCommit, CodeBuild & CodeDeploy Next Chapter: Chapter 33: AWS CloudFormation - Infrastructure as Code