Storage_gateway
Chapter 20: AWS Storage Gateway - Hybrid Storage
Section titled “Chapter 20: AWS Storage Gateway - Hybrid Storage”Bridging On-Premises and Cloud Storage
Section titled “Bridging On-Premises and Cloud Storage”20.1 Overview
Section titled “20.1 Overview”AWS Storage Gateway is a hybrid storage service that enables your on-premises applications to seamlessly use AWS cloud storage.
Storage Gateway Overview+------------------------------------------------------------------+| || +------------------------+ || | Storage Gateway | || +------------------------+ || | || +---------------------+---------------------+ || | | | || v v v || +----------+ +----------+ +----------+ || | S3 File | | FSx File | | Volume | || | Gateway | | Gateway | | Gateway | || | | | | | | || | - S3 | | - FSx | | - EBS | || | storage| | Windows| | snapshot| || | - SMB/NFS| | File | | - iSCSI | || | | | Server | | | || +----------+ +----------+ +----------+ || | | | || v v v || +----------+ +----------+ +----------+ || | Tape | | | | Cached | || | Gateway | | | | Volume | || | | | | | Gateway | || | - Virtual| | | | | || | tapes | | | | | || | - S3/Glacier| | | | | || +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+20.2 Gateway Types
Section titled “20.2 Gateway Types”S3 File Gateway
Section titled “S3 File Gateway” S3 File Gateway+------------------------------------------------------------------+| || Architecture || +----------------------------------------------------------+ || | | || | On-Premises AWS Cloud | || | +------------------+ +------------------+ | || | | | | | | || | | +------------+ | | +------------+ | | || | | | Application| | | | S3 Bucket | | | || | | +------------+ | | | | | | || | | | | | +------------+ | | || | | v | | ^ | | || | | +------------+ | | | | | || | | | S3 File | | | +------------+ | | || | | | Gateway |----+----->| Storage | | | || | | | (SMB/NFS) | | | | Gateway | | | || | | +------------+ | | +------------+ | | || | | | | | | | || | | v | | | | || | | +------------+ | | | | || | | | Local Cache| | | | | || | | +------------+ | | | | || | | | | | | || | +------------------+ +------------------+ | || | | || +----------------------------------------------------------+ || || Features: || - SMB or NFS interface || - Maps to S3 buckets || - Local cache for low-latency access || - Transparent S3 integration || || Use Cases: || - File shares backed by S3 || - Backup and archive || - Data migration to S3 || |+------------------------------------------------------------------+FSx File Gateway
Section titled “FSx File Gateway” FSx File Gateway+------------------------------------------------------------------+| || Architecture || +----------------------------------------------------------+ || | | || | On-Premises AWS Cloud | || | +------------------+ +------------------+ | || | | | | | | || | | +------------+ | | +------------+ | | || | | | Application| | | | FSx for | | | || | | +------------+ | | | Windows | | | || | | | | | +------------+ | | || | | v | | ^ | | || | | +------------+ | | | | | || | | | FSx File | | | +------------+ | | || | | | Gateway |----+----->| Storage | | | || | | | (SMB) | | | | Gateway | | | || | | +------------+ | | +------------+ | | || | | | | | | | || | | v | | | | || | | +------------+ | | | | || | | | Local Cache| | | | | || | | +------------+ | | | | || | | | | | | || | +------------------+ +------------------+ | || | | || +----------------------------------------------------------+ || || Features: || - SMB interface || - Windows file server in cloud || - Active Directory integration || - Local cache for frequently accessed files || || Use Cases: || - Windows file shares || - Home directories || - Application data || |+------------------------------------------------------------------+Volume Gateway
Section titled “Volume Gateway” Volume Gateway+------------------------------------------------------------------+| || Cached Volume Mode || +----------------------------------------------------------+ || | | || | On-Premises AWS Cloud | || | +------------------+ +------------------+ | || | | | | | | || | | +------------+ | | +------------+ | | || | | | Application| | | | EBS | | | || | | | (iSCSI) | | | | Snapshots | | | || | | +------------+ | | +------------+ | | || | | | | | ^ | | || | | v | | | | | || | | +------------+ | | +------------+ | | || | | | Volume | | | | S3 Bucket | | | || | | | Gateway |----+----->| (Data) | | | || | | | (Cached) | | | +------------+ | | || | | +------------+ | | | | || | | | | | | | || | | v | | | | || | | +------------+ | | | | || | | | Local Cache| | | | | || | | | (Hot Data) | | | | | || | | +------------+ | | | | || | | | | | | || | +------------------+ +------------------+ | || | | || | Primary data in S3, cache on-premises | || | | || +----------------------------------------------------------+ || || Stored Volume Mode || +----------------------------------------------------------+ || | | || | On-Premises AWS Cloud | || | +------------------+ +------------------+ | || | | | | | | || | | +------------+ | | +------------+ | | || | | | Application| | | | EBS | | | || | | | (iSCSI) | | | | Snapshots | | | || | | +------------+ | | +------------+ | | || | | | | | ^ | | || | | v | | | | | || | | +------------+ | | +------------+ | | || | | | Volume | | | | S3 Bucket | | | || | | | Gateway |----+----->| (Backup) | | | || | | | (Stored) | | | +------------+ | | || | | +------------+ | | | | || | | | | | | | || | | v | | | | || | | +------------+ | | | | || | | | Local Disk | | | | | || | | | (All Data) | | | | | || | | +------------+ | | | | || | | | | | | || | +------------------+ +------------------+ | || | | || | Primary data on-premises, async backup to S3 | || | | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+Tape Gateway
Section titled “Tape Gateway” Tape Gateway+------------------------------------------------------------------+| || Architecture || +----------------------------------------------------------+ || | | || | On-Premises AWS Cloud | || | +------------------+ +------------------+ | || | | | | | | || | | +------------+ | | +------------+ | | || | | | Backup | | | | S3 Glacier | | | || | | | Software | | | | (Archive) | | | || | | +------------+ | | +------------+ | | || | | | | | ^ | | || | | v | | | | | || | | +------------+ | | +------------+ | | || | | | Tape | | | | S3 Bucket | | | || | | | Gateway |----+----->| (Tapes) | | | || | | | (iSCSI) | | | +------------+ | | || | | +------------+ | | | | || | | | | | | | || | | v | | | | || | | +------------+ | | | | || | | | Virtual | | | | | || | | | Tapes | | | | | || | | | (VTL) | | | | | || | | +------------+ | | | | || | | | | | | || | +------------------+ +------------------+ | || | | || +----------------------------------------------------------+ || || Features: || - iSCSI VTL (Virtual Tape Library) || - Compatible with backup software || - Automatic tiering to Glacier || - Cost-effective tape backup || || Use Cases: || - Backup and archive || - Replace physical tape infrastructure || - Compliance and retention || |+------------------------------------------------------------------+20.3 Gateway Deployment Options
Section titled “20.3 Gateway Deployment Options” Gateway Deployment Options+------------------------------------------------------------------+| || 1. Hardware Appliance || +----------------------------------------------------------+ || | | || | Features: | || | - Physical device | || | - Pre-configured | || | - Includes CPU, memory, SSD cache | || | | || | Use Case: | || | - Production workloads | || | - High performance requirements | || | | || +----------------------------------------------------------+ || || 2. Virtual Machine || +----------------------------------------------------------+ || | | || | Platforms: | || | - VMware ESXi | || | - Microsoft Hyper-V | || | - Linux KVM | || | | || | Requirements: | || | - 4+ vCPUs | || | - 16+ GB RAM | || | - Local storage for cache | || | | || +----------------------------------------------------------+ || || 3. Amazon EC2 || +----------------------------------------------------------+ || | | || | Features: | || | - Deploy on EC2 instance | || | - Use in AWS region | || | - AMI provided | || | | || | Use Case: | || | - Testing and development | || | - Cloud-based applications | || | | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+20.4 Practical Configuration
Section titled “20.4 Practical Configuration”Storage Gateway with Terraform
Section titled “Storage Gateway with Terraform”# ============================================================# Storage Gateway# ============================================================
# Activation Key (required for gateway activation)# This is typically done through the console or AWS CLI
# ============================================================# S3 File Gateway# ============================================================
resource "aws_storagegateway_gateway" "s3" { gateway_name = "s3-file-gateway" gateway_timezone = "GMT" gateway_type = "FILE_S3"
# Activation activation_key = var.activation_key
# IP address (on-premises gateway) gateway_ip_address = "192.168.1.100"
# CloudWatch logging cloudwatch_log_group_arn = aws_cloudwatch_log_group.gateway.arn
tags = { Name = "s3-file-gateway" }}
# S3 File Shareresource "aws_storagegateway_nfs_file_share" "main" { gateway_arn = aws_storagegateway_gateway.s3.arn location_arn = aws_s3_bucket.data.arn role_arn = aws_iam_role.gateway.arn
# NFS settings default_storage_class = "S3_STANDARD"
# Squash settings squash = "ROOT_SQUASH"
# Export options export { read_only = false squash_option = "ROOT_SQUASH" }
tags = { Name = "nfs-file-share" }}
# SMB File Shareresource "aws_storagegateway_smb_file_share" "main" { gateway_arn = aws_storagegateway_gateway.s3.arn location_arn = aws_s3_bucket.data.arn role_arn = aws_iam_role.gateway.arn
# SMB settings default_storage_class = "S3_STANDARD"
# Authentication authentication = "ActiveDirectory"
# Access control admin_user_list = ["Admin"] valid_user_list = ["User1", "User2"]
tags = { Name = "smb-file-share" }}
# ============================================================# FSx File Gateway# ============================================================
resource "aws_storagegateway_gateway" "fsx" { gateway_name = "fsx-file-gateway" gateway_timezone = "GMT" gateway_type = "FILE_FSX_SMB"
activation_key = var.activation_key
tags = { Name = "fsx-file-gateway" }}
# FSx File Shareresource "aws_storagegateway_smb_file_share" "fsx" { gateway_arn = aws_storagegateway_gateway.fsx.arn location_arn = aws_fsx_windows_file_system.main.arn role_arn = aws_iam_role.gateway.arn
tags = { Name = "fsx-smb-share" }}
# ============================================================# Volume Gateway (Cached)# ============================================================
resource "aws_storagegateway_gateway" "volume" { gateway_name = "volume-gateway" gateway_timezone = "GMT" gateway_type = "STORED" # or "CACHED"
activation_key = var.activation_key
tags = { Name = "volume-gateway" }}
# Cached Volumeresource "aws_storagegateway_cached_iscsi_volume" "main" { gateway_arn = aws_storagegateway_gateway.volume.arn volume_size_in_bytes = 107374182400 # 100 GB
# Source from snapshot (optional) # snapshot_id = aws_ebs_snapshot.main.id
tags = { Name = "cached-volume" }}
# Stored Volumeresource "aws_storagegateway_stored_iscsi_volume" "main" { gateway_arn = aws_storagegateway_gateway.volume.arn disk_id = "disk-1" # Local disk ID volume_size_in_bytes = 107374182400 # 100 GB
tags = { Name = "stored-volume" }}
# ============================================================# Tape Gateway# ============================================================
resource "aws_storagegateway_gateway" "tape" { gateway_name = "tape-gateway" gateway_timezone = "GMT" gateway_type = "VTL"
activation_key = var.activation_key
tags = { Name = "tape-gateway" }}
# Virtual Taperesource "aws_storagegateway_tape" "main" { gateway_arn = aws_storagegateway_gateway.tape.arn tape_size_in_bytes = 107374182400 # 100 GB
tags = { Name = "virtual-tape" }}
# Tape Pool (for archiving)resource "aws_storagegateway_tape_pool" "archive" { pool_name = "archive-pool"
# Retention lock retention_lock_type = "LOCKED" retention_lock_time_in_days = 365
tags = { Name = "archive-pool" }}
# ============================================================# IAM Role for Storage Gateway# ============================================================
resource "aws_iam_role" "gateway" { name = "storage-gateway-role"
assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = "sts:AssumeRole" Effect = "Allow" Principal = { Service = "storagegateway.amazonaws.com" } } ] })}
resource "aws_iam_role_policy" "gateway" { name = "storage-gateway-policy" role = aws_iam_role.gateway.id
policy = jsonencode({ Version = "2012-10-17" Statement = [ { Effect = "Allow" Action = [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket", "s3:GetBucketLocation" ] Resource = [ aws_s3_bucket.data.arn, "${aws_s3_bucket.data.arn}/*" ] } ] })}
# ============================================================# CloudWatch Log Group# ============================================================
resource "aws_cloudwatch_log_group" "gateway" { name = "/aws/storagegateway/main" retention_in_days = 30}20.5 Gateway Comparison
Section titled “20.5 Gateway Comparison” Gateway Type Comparison+------------------------------------------------------------------+| || Feature | S3 File | FSx File | Volume | Tape || ---------------|-------------|-------------|-----------|--------|| Protocol | SMB/NFS | SMB | iSCSI | iSCSI || Storage | S3 | FSx Windows | S3/EBS | S3 || Interface | File | File | Block | Tape || Cache | Yes | Yes | Yes | No || Snapshots | No | No | Yes | No || Archive | Lifecycle | No | No | Glacier|| ---------------|-------------|-------------|-----------|--------|| Use Case | File shares | Windows | Block | Backup || | Backup | Home dirs | storage | Archive|| |+------------------------------------------------------------------+20.6 Best Practices
Section titled “20.6 Best Practices” Storage Gateway Best Practices+------------------------------------------------------------------+| || 1. Network Configuration || +----------------------------------------------------------+ || | - Ensure sufficient bandwidth | || | - Use Direct Connect for large data | || | - Configure QoS for gateway traffic | || +----------------------------------------------------------+ || || 2. Cache Configuration || +----------------------------------------------------------+ || | - Size cache based on working set | || | - Use SSD for cache storage | || | - Monitor cache hit ratio | || +----------------------------------------------------------+ || || 3. Security || +----------------------------------------------------------+ || | - Use HTTPS for communication | || | - Configure IAM policies | || | - Enable encryption at rest | || | - Use Active Directory for SMB | || +----------------------------------------------------------+ || || 4. Monitoring || +----------------------------------------------------------+ || | - Enable CloudWatch logging | || | - Monitor cache usage | || | - Set up alerts for issues | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+20.7 Exam Tips
Section titled “20.7 Exam Tips”
- S3 File Gateway: SMB/NFS interface, S3 backend, local cache
- FSx File Gateway: SMB interface, FSx Windows backend
- Volume Gateway: iSCSI block storage, cached or stored mode
- Tape Gateway: Virtual tape library, Glacier archive
- Cached Volume: Primary in S3, cache on-premises
- Stored Volume: Primary on-premises, async backup to S3
- Deployment: Hardware appliance, VM, or EC2
- Activation Key: Required to activate gateway
- Cache: Use SSD, size based on working set
- Network: Direct Connect recommended for large data
Next Chapter
Section titled “Next Chapter”Chapter 21: Amazon RDS - Relational Database Service
Last Updated: February 2026