Secrets_management
Chapter 27: AWS KMS, Secrets Manager & Parameter Store
Section titled “Chapter 27: AWS KMS, Secrets Manager & Parameter Store”Secrets Management & Encryption Services
Section titled “Secrets Management & Encryption Services”27.1 Overview
Section titled “27.1 Overview”AWS provides multiple services for managing secrets, encryption keys, and configuration data. Understanding when to use each service is crucial for building secure applications.
AWS Secrets Management Overview+------------------------------------------------------------------+| || +------------------------+ || | Secrets Management | || +------------------------+ || | || +---------------------+---------------------+ || | | | || v v v || +----------+ +----------+ +----------+ || | KMS | | Secrets | | Parameter| || | | | Manager | | Store | || | - Keys | | - Secrets| | - Config | || | - Encrypt| | - Rotate | | - Params | || | - Sign | | - Version| | - Hierarchy| || +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+Service Comparison
Section titled “Service Comparison”| Feature | KMS | Secrets Manager | Parameter Store |
|---|---|---|---|
| Primary Use | Encryption keys | Database credentials | Configuration data |
| Auto Rotation | No | Yes | No |
| Versioning | Key versions | Secret versions | Parameter history |
| Pricing | Per API call | Per secret/month | Free (Standard) |
| Max Size | N/A | 10 KB | 4 KB (Standard) / 8 KB (Advanced) |
| Encryption | Manages keys | Uses KMS | Uses KMS |
27.2 AWS KMS (Key Management Service)
Section titled “27.2 AWS KMS (Key Management Service)”KMS Architecture
Section titled “KMS Architecture” AWS KMS Architecture+------------------------------------------------------------------+| || +------------------------+ || | AWS KMS | || +------------------------+ || | || +---------------------+---------------------+ || | | | || v v v || +----------+ +----------+ +----------+ || | Customer | | AWS | | Custom | || | Managed | | Managed | | Key Store| || | Keys | | Keys | | | || | (CMK) | | (AWS) | | - HSM | || | | | | | - CloudHSM || | - Create | | - S3 | | | || | - Rotate | | - EBS | | | || | - Delete | | - RDS | | | || +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+Key Types
Section titled “Key Types” KMS Key Types+------------------------------------------------------------------+| || Customer Managed Keys (CMK) || +------------------------------------------------------------+ || | - Full control over key policies | || | - Can enable/disable key rotation | || | - Can schedule key deletion | || | - Support for key material import | || | - Cost: $1/month + API charges | || +------------------------------------------------------------+ || || AWS Managed Keys || +------------------------------------------------------------+ || | - Created and managed by AWS services | || | - Auto-rotated every 3 years | || | - Cannot delete or manage policies | || | - No monthly fee | || | - Examples: aws/s3, aws/rds, aws/ebs | || +------------------------------------------------------------+ || || AWS Owned Keys || +------------------------------------------------------------+ || | - Used by some AWS services (S3, SNS, etc.) | || | - Not visible in your account | || | - No management required | || | - No cost | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Key Policies
Section titled “Key Policies” KMS Key Policy Structure+------------------------------------------------------------------+| || Default Key Policy || +------------------------------------------------------------+ || | { | || | "Version": "2012-10-17", | || | "Statement": [ | || | { | || | "Sid": "Enable IAM User Permissions", | || | "Effect": "Allow", | || | "Principal": { | || | "AWS": "arn:aws:iam::123456789012:root" | || | }, | || | "Action": "kms:*", | || | "Resource": "*" | || | } | || | ] | || | } | || +------------------------------------------------------------+ || || Custom Key Policy (Cross-Account Access) || +------------------------------------------------------------+ || | { | || | "Version": "2012-10-17", | || | "Statement": [ | || | { | || | "Sid": "Allow use of the key", | || | "Effect": "Allow", | || | "Principal": { | || | "AWS": "arn:aws:iam::999999999999:root" | || | }, | || | "Action": [ | || | "kms:Encrypt", | || | "kms:Decrypt", | || | "kms:ReEncrypt*", | || | "kms:GenerateDataKey*", | || | "kms:DescribeKey" | || | ], | || | "Resource": "*" | || | } | || | ] | || | } | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Encryption Patterns
Section titled “Encryption Patterns” KMS Encryption Patterns+------------------------------------------------------------------+| || Envelope Encryption (Most Common) || +------------------------------------------------------------+ || | | || | 1. Application requests data key from KMS | || | +--------+ +--------+ | || | | App | --> | KMS | | || | +--------+ +--------+ | || | | | || | v | || | +----------+ | || | | Data Key | | || | | (Plain) | | || | | + Encrypted | || | +----------+ | || | | | || | 2. Encrypt data with plaintext key | || | +--------+ +----------+ +--------+ | || | | Data | --> | Data Key | --> | Cipher | | || | +--------+ +----------+ +--------+ | || | | | || | 3. Store encrypted data key with encrypted data | || | +------------------------------------------+ | || | | Encrypted Data + Encrypted Data Key | | || | +------------------------------------------+ | || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+KMS CLI Commands
Section titled “KMS CLI Commands”# Create keyaws kms create-key \ --description "My encryption key" \ --key-usage ENCRYPT_DECRYPT \ --origin AWS_KMS
# Create aliasaws kms create-alias \ --alias-name alias/my-key \ --target-key-id 1234abcd-12ab-34cd-56ef-1234567890ab
# Encryptaws kms encrypt \ --key-id alias/my-key \ --plaintext fileb://plaintext.txt \ --output text \ --query CiphertextBlob | base64 --decode > encrypted.txt
# Decryptaws kms decrypt \ --ciphertext-blob fileb://encrypted.txt \ --output text \ --query Plaintext | base64 --decode > decrypted.txt
# Generate data key (for envelope encryption)aws kms generate-data-key \ --key-id alias/my-key \ --key-spec AES_256
# Enable key rotationaws kms enable-key-rotation \ --key-id 1234abcd-12ab-34cd-56ef-1234567890ab
# Schedule key deletionaws kms schedule-key-deletion \ --key-id 1234abcd-12ab-34cd-56ef-1234567890ab \ --pending-window-in-days 3027.3 AWS Secrets Manager
Section titled “27.3 AWS Secrets Manager”Secrets Manager Architecture
Section titled “Secrets Manager Architecture” AWS Secrets Manager Architecture+------------------------------------------------------------------+| || +------------------------+ || | Secrets Manager | || +------------------------+ || | || +---------------------+---------------------+ || | | | || v v v || +----------+ +----------+ +----------+ || | Database | | Third | | Custom | || | Secrets | | Party | | Secrets | || | | | API Keys | | | || | - RDS | | | | - API | || | - DocDB | | - API | | Keys | || | - Redshift| | - Tokens | | - Certs | || +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+Secret Types
Section titled “Secret Types” Secrets Manager Secret Types+------------------------------------------------------------------+| || Database Credentials || +------------------------------------------------------------+ || | { | || | "username": "admin", | || | "password": "MySecurePassword123!", | || | "engine": "mysql", | || | "host": "mydb.cluster-xxx.region.rds.amazonaws.com", | || | "port": 3306, | || | "dbname": "mydb" | || | } | || +------------------------------------------------------------+ || || API Keys / Tokens || +------------------------------------------------------------+ || | { | || | "api_key": "sk-xxxxxxxxxxxxxxxx", | || | "api_secret": "xxxxxxxxxxxxxxxx", | || | "endpoint": "https://api.example.com" | || | } | || +------------------------------------------------------------+ || || Custom Secrets || +------------------------------------------------------------+ || | { | || | "certificate": "-----BEGIN CERTIFICATE-----...", | || | "private_key": "-----BEGIN PRIVATE KEY-----...", | || | "chain": "-----BEGIN CERTIFICATE-----..." | || | } | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Automatic Rotation
Section titled “Automatic Rotation” Secrets Manager Rotation+------------------------------------------------------------------+| || Rotation Workflow || +------------------------------------------------------------+ || | | || | +--------+ +--------+ +--------+ +--------+ | || | | Secret | --> | Lambda | --> | Update | --> | Update | | || | | Version| | Function| | Database| | Secret | | || | | 1 | | | | Password| | Version| | || | +--------+ +--------+ +--------+ +--------+ | || | | || | Rotation Configuration: | || | - Rotation interval: 1-365 days | || | - Lambda function: Custom or AWS provided | || | - Rotation strategy: Single user or alternating | || | | || +------------------------------------------------------------+ || || Built-in Rotation Templates || +------------------------------------------------------------+ || | - RDS MySQL/PostgreSQL/Oracle/SQL Server | || | - Amazon DocumentDB | || | - Amazon Redshift | || | - Amazon Aurora | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Secrets Manager CLI Commands
Section titled “Secrets Manager CLI Commands”# Create secretaws secretsmanager create-secret \ --name "prod/database/credentials" \ --description "Production database credentials" \ --secret-string '{"username":"admin","password":"MyPassword123"}'
# Get secret valueaws secretsmanager get-secret-value \ --secret-id "prod/database/credentials"
# Update secretaws secretsmanager put-secret-value \ --secret-id "prod/database/credentials" \ --secret-string '{"username":"admin","password":"NewPassword456"}'
# Enable rotationaws secretsmanager rotate-secret \ --secret-id "prod/database/credentials" \ --rotation-lambda-arn arn:aws:lambda:us-east-1:123456789012:function:MyRotationFunction \ --rotation-rules AutomaticallyAfterDays=30
# List secretsaws secretsmanager list-secrets
# Delete secret (with recovery window)aws secretsmanager delete-secret \ --secret-id "prod/database/credentials" \ --recovery-window-in-days 7
# Restore secretaws secretsmanager restore-secret \ --secret-id "prod/database/credentials"27.4 AWS Systems Manager Parameter Store
Section titled “27.4 AWS Systems Manager Parameter Store”Parameter Store Architecture
Section titled “Parameter Store Architecture” Parameter Store Architecture+------------------------------------------------------------------+| || +------------------------+ || | Parameter Store | || +------------------------+ || | || +---------------------+---------------------+ || | | | || v v v || +----------+ +----------+ +----------+ || | String | | StringList| | SecureString || | | | | | | || | - Plain | | - Comma | | - Encrypted| || | Text | | Separated| | - KMS | || | - Config | | - Lists | | - Secrets| || +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+Parameter Hierarchy
Section titled “Parameter Hierarchy” Parameter Store Hierarchy+------------------------------------------------------------------+| || Recommended Hierarchy Structure || +------------------------------------------------------------+ || | | || | / | || | |-- /my-app | || | |-- /prod | || | |-- /database | || | | |-- /host | || | | |-- /port | || | | |-- /username | || | | |-- /password (SecureString) | || | |-- /api | || | |-- /endpoint | || | |-- /key (SecureString) | || | |-- /dev | || | |-- /database | || | |-- /host | || | |-- /password (SecureString) | || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Parameter Tiers
Section titled “Parameter Tiers” Parameter Store Tiers+------------------------------------------------------------------+| || Standard Tier (Free) || +------------------------------------------------------------+ || | - Max parameter size: 4 KB | || | - Parameter versions: 100 | || | - Throughput: Standard (default) | || | - Cost: FREE | || +------------------------------------------------------------+ || || Advanced Tier (Paid) || +------------------------------------------------------------+ || | - Max parameter size: 8 KB | || | - Parameter versions: 1000 | || | - Throughput: Higher available | || | - Cost: $0.05/parameter/month + API charges | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Parameter Store CLI Commands
Section titled “Parameter Store CLI Commands”# Put parameter (String)aws ssm put-parameter \ --name "/my-app/prod/database/host" \ --value "db.example.com" \ --type String \ --description "Database host"
# Put parameter (SecureString)aws ssm put-parameter \ --name "/my-app/prod/database/password" \ --value "MySecurePassword123" \ --type SecureString \ --key-id alias/my-key \ --description "Database password"
# Get parameteraws ssm get-parameter \ --name "/my-app/prod/database/host"
# Get parameter (with decryption)aws ssm get-parameter \ --name "/my-app/prod/database/password" \ --with-decryption
# Get parameters by pathaws ssm get-parameters-by-path \ --path "/my-app/prod/database" \ --recursive \ --with-decryption
# Get multiple parametersaws ssm get-parameters \ --names "/my-app/prod/database/host" "/my-app/prod/database/port" \ --with-decryption
# Update parameteraws ssm put-parameter \ --name "/my-app/prod/database/host" \ --value "new-db.example.com" \ --type String \ --overwrite
# Delete parameteraws ssm delete-parameter \ --name "/my-app/prod/database/host"
# List parametersaws ssm describe-parameters27.5 Service Selection Guide
Section titled “27.5 Service Selection Guide”When to Use Each Service
Section titled “When to Use Each Service” Service Selection Decision Tree+------------------------------------------------------------------+| || Need to store secrets? || | || v || +------------------------+ || | Need automatic | || | rotation? | || +------------------------+ || | | || Yes No || | | || v v || +----------+ +------------------------+ || | Secrets | | Need encryption? | || | Manager | +------------------------+ || +----------+ | | || Yes No || | | || v v || +----------+ +----------+ || | Parameter | | Parameter | || | Store | | Store | || | (Secure) | | (String) | || +----------+ +----------+ || |+------------------------------------------------------------------+Use Case Matrix
Section titled “Use Case Matrix” Use Case Matrix+------------------------------------------------------------------+| || Use Case | Recommended Service || ----------------------------|----------------------------------- || Database credentials | Secrets Manager (with rotation) || API keys | Secrets Manager or Parameter Store|| Application config | Parameter Store || Encryption keys | KMS || SSH keys | Parameter Store (SecureString) || TLS certificates | Secrets Manager or ACM || Feature flags | Parameter Store || Connection strings | Parameter Store || Third-party tokens | Secrets Manager (with rotation) || |+------------------------------------------------------------------+27.6 Integration with AWS Services
Section titled “27.6 Integration with AWS Services”EC2 Integration
Section titled “EC2 Integration” EC2 Secret Retrieval+------------------------------------------------------------------+| || Using IAM Role + Parameter Store || +------------------------------------------------------------+ || | | || | +--------+ +--------+ +----------+ | || | | EC2 | --> | IAM | --> | Parameter| | || | | Instance| | Role | | Store | | || | +--------+ +--------+ +----------+ | || | | | | || | v v | || | +----------+ +----------+ | || | | User Data| | Secrets | | || | | Script | | Retrieved| | || | +----------+ +----------+ | || | | || | #!/bin/bash | || | DB_PASS=$(aws ssm get-parameter \ | || | --name "/app/db/password" \ | || | --with-decryption \ | || | --query Parameter.Value --output text) | || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Lambda Integration
Section titled “Lambda Integration” Lambda Secret Retrieval+------------------------------------------------------------------+| || Using Lambda Environment Variables + Secrets Manager || +------------------------------------------------------------+ || | | || | +--------+ +--------+ +----------+ | || | | Lambda | --> | Env Var| --> | Secrets | | || | | | | (ARN) | | Manager | | || | +--------+ +--------+ +----------+ | || | | || | # Python example | || | import boto3 | || | import os | || | | || | client = boto3.client('secretsmanager') | || | secret_arn = os.environ['SECRET_ARN'] | || | | || | response = client.get_secret_value(SecretId=secret_arn) | || | secret = json.loads(response['SecretString']) | || | | || +------------------------------------------------------------+ || || Using Lambda Environment Variables + Parameter Store || +------------------------------------------------------------+ || | | || | # Lambda console: Environment Variables | || | DB_HOST = {{resolve:ssm:/app/db/host:1}} | || | DB_PASS = {{resolve:ssm-secure:/app/db/password:1}} | || | | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+CloudFormation Integration
Section titled “CloudFormation Integration”# CloudFormation with Parameter StoreParameters: DatabasePassword: Type: AWS::SSM::Parameter::Value<String> Default: /app/prod/database/password NoEcho: true
Resources: MyDatabase: Type: AWS::RDS::DBInstance Properties: MasterUserPassword: !Ref DatabasePassword
# CloudFormation with Secrets ManagerResources: MySecret: Type: AWS::SecretsManager::Secret Properties: Name: prod/database/credentials GenerateSecretString: SecretStringTemplate: '{"username": "admin"}' GenerateStringKey: "password" PasswordLength: 32 ExcludeCharacters: '"@/\'
MyDatabase: Type: AWS::RDS::DBInstance Properties: MasterUsername: !Sub '{{resolve:secretsmanager:${MySecret}:SecretString:username}}' MasterUserPassword: !Sub '{{resolve:secretsmanager:${MySecret}:SecretString:password}}'27.7 Security Best Practices
Section titled “27.7 Security Best Practices”KMS Best Practices
Section titled “KMS Best Practices” KMS Security Best Practices+------------------------------------------------------------------+| || 1. Use least privilege key policies || +------------------------------------------------------------+ || | - Grant only required actions | || | - Use IAM conditions for additional restrictions | || +------------------------------------------------------------+ || || 2. Enable automatic key rotation || +------------------------------------------------------------+ || | - Customer managed keys: Enable rotation | || | - AWS managed keys: Auto-rotated every 3 years | || +------------------------------------------------------------+ || || 3. Use key aliases for easier management || +------------------------------------------------------------+ || | - alias/prod-key instead of key-id | || | - Easier to update keys without code changes | || +------------------------------------------------------------+ || || 4. Monitor KMS usage with CloudTrail || +------------------------------------------------------------+ || | - All KMS API calls are logged | || | - Monitor for unauthorized access attempts | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Secrets Manager Best Practices
Section titled “Secrets Manager Best Practices” Secrets Manager Best Practices+------------------------------------------------------------------+| || 1. Enable automatic rotation || +------------------------------------------------------------+ || | - Rotate database credentials regularly | || | - Use built-in rotation templates when possible | || +------------------------------------------------------------+ || || 2. Use resource-based policies || +------------------------------------------------------------+ || | - Limit access to specific secrets | || | - Use conditions for additional security | || +------------------------------------------------------------+ || || 3. Monitor secret access || +------------------------------------------------------------+ || | - Use CloudTrail to track secret access | || | - Set up alerts for unusual access patterns | || +------------------------------------------------------------+ || || 4. Use versioning || +------------------------------------------------------------+ || | - Track changes to secrets | || | - Ability to rollback if needed | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Parameter Store Best Practices
Section titled “Parameter Store Best Practices” Parameter Store Best Practices+------------------------------------------------------------------+| || 1. Use hierarchical naming || +------------------------------------------------------------+ || | - /environment/application/service/parameter | || | - Enables bulk retrieval by path | || +------------------------------------------------------------+ || || 2. Use SecureString for sensitive data || +------------------------------------------------------------+ || | - Encrypt passwords, API keys, tokens | || | - Use customer-managed KMS keys | || +------------------------------------------------------------+ || || 3. Use IAM policies for access control || +------------------------------------------------------------+ || | - Grant access to specific paths | || | - Use conditions for additional restrictions | || +------------------------------------------------------------+ || || 4. Tag parameters for organization || +------------------------------------------------------------+ || | - Use tags for cost allocation | || | - Use tags for automation and filtering | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+27.8 Troubleshooting
Section titled “27.8 Troubleshooting”Common Issues
Section titled “Common Issues” Common Issues and Solutions+------------------------------------------------------------------+| || Issue 1: Access Denied when accessing secret || +------------------------------------------------------------+ || | Cause: Missing IAM permissions | || | Solution: Add secretsmanager:GetSecretValue to IAM policy | || +------------------------------------------------------------+ || || Issue 2: Cannot decrypt SecureString || +------------------------------------------------------------+ || | Cause: No access to KMS key | || | Solution: Add kms:Decrypt permission for the KMS key | || +------------------------------------------------------------+ || || Issue 3: Secret rotation failed || +------------------------------------------------------------+ || | Cause: Lambda function permissions or network issues | || | Solution: Check Lambda logs and VPC configuration | || +------------------------------------------------------------+ || || Issue 4: Parameter size exceeded || +------------------------------------------------------------+ || | Cause: Parameter value > 4 KB (Standard tier) | || | Solution: Use Advanced tier or split into multiple params | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+27.9 Exam Tips
Section titled “27.9 Exam Tips”
Key Exam Points+------------------------------------------------------------------+| || 1. KMS is for encryption keys, not for storing secrets || || 2. Secrets Manager = automatic rotation (database credentials) || || 3. Parameter Store = configuration data (free for Standard) || || 4. Use SecureString for encrypted parameters || || 5. KMS key policies are resource-based (not IAM) || || 6. Envelope encryption = data key + encrypted data key || || 7. AWS managed keys auto-rotate every 3 years || || 8. Customer managed keys can be manually or auto-rotated || || 9. Secrets Manager costs $0.40/secret/month + API calls || || 10. Parameter Store Standard tier is FREE || |+------------------------------------------------------------------+27.10 Summary
Section titled “27.10 Summary” Chapter 27 Summary+------------------------------------------------------------------+| || AWS KMS || +------------------------------------------------------------+ || | - Manages encryption keys | || | - Customer managed vs AWS managed keys | || | - Key policies for access control | || | - Envelope encryption for large data | || +------------------------------------------------------------+ || || AWS Secrets Manager || +------------------------------------------------------------+ || | - Store and rotate secrets | || | - Built-in database credential rotation | || | - Versioning and recovery | || | - Integration with RDS, Lambda, EC2 | || +------------------------------------------------------------+ || || AWS Parameter Store || +------------------------------------------------------------+ || | - Configuration management | || | - Hierarchical parameter names | || | - String, StringList, SecureString types | || | - Free (Standard tier) | || +------------------------------------------------------------+ || |+------------------------------------------------------------------+Previous Chapter: Chapter 26: AWS Organizations & Service Control Policies Next Chapter: Chapter 28: AWS WAF, Shield & Firewall