Amazon FSx - Managed File Systems
Chapter 19: Amazon FSx - File Storage Solutions
Section titled “Chapter 19: Amazon FSx - File Storage Solutions”Managed File Systems for Enterprise Workloads
Section titled “Managed File Systems for Enterprise Workloads”19.1 Overview
Section titled “19.1 Overview”Amazon FSx provides fully managed third-party file systems optimized for a wide range of enterprise and high-performance computing (HPC) workloads.
FSx Overview+------------------------------------------------------------------+| || +------------------------+ || | Amazon FSx | || +------------------------+ || | || +---------------------+---------------------+ || | | | || v v v || +----------+ +----------+ +----------+ || | FSx for | | FSx for | | FSx for | || | Windows | | Lustre | | NetApp | || | File | | | | ONTAP | || | Server | | | | | || +----------+ +----------+ +----------+ || | | | || v v v || +----------+ +----------+ +----------+ || | SMB | | HPC | | Multi- | || | Windows | | ML/AI | | protocol | || | Active | | High | | NFS/SMB | || | Directory| | Perf | | iSCSI | || +----------+ +----------+ +----------+ || |+------------------------------------------------------------------+19.2 FSx for Windows File Server
Section titled “19.2 FSx for Windows File Server”Architecture
Section titled “Architecture” FSx for Windows File Server+------------------------------------------------------------------+| || Architecture || +----------------------------------------------------------+ || | | || | +----------------------------------------------------+ | || | | FSx File System | | || | | (Multi-AZ or Single-AZ) | | || | +----------------------------------------------------+ | || | | | || | +-------------+-------------+ | || | | | | | || | v v v | || | +----------+ +----------+ +----------+ | || | | Windows | | Windows | | Windows | | || | | Server 1 | | Server 2 | | Server N | | || | +----------+ +----------+ +----------+ | || | | | | | || | v v v | || | +----------------------------------------------------+ | || | | Active Directory | | || | | (AWS Managed or Self-Managed) | | || | +----------------------------------------------------+ | || | | || +----------------------------------------------------------+ || || Features: || - Native Windows file system (SMB) || - Active Directory integration || - DFS Namespaces support || - Windows-native backup and restore || - VSS (Volume Shadow Copy Service) || |+------------------------------------------------------------------+Deployment Types
Section titled “Deployment Types” FSx for Windows Deployment Types+------------------------------------------------------------------+| || Single-AZ || +----------------------------------------------------------+ || | | || | +------------------+ | || | | Availability | | || | | Zone A | | || | | +------------+ | | || | | | FSx File | | | || | | | System | | | || | | +------------+ | | || | +------------------+ | || | | || | Use Case: | || | - Development environments | || | - Non-critical workloads | || | - Cost optimization | || | | || +----------------------------------------------------------+ || || Multi-AZ || +----------------------------------------------------------+ || | | || | +------------------+ +------------------+ | || | | Availability | | Availability | | || | | Zone A | | Zone B | | || | | +------------+ | | +------------+ | | || | | | FSx Primary| |<-->| | FSx Standby| | | || | | +------------+ | | +------------+ | | || | +------------------+ +------------------+ | || | | || | Features: | || | - Automatic failover | || | - High availability | || | - Synchronous replication | || | | || | Use Case: | || | - Production workloads | || | - Mission-critical applications | || | | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+Storage Capacity and Performance
Section titled “Storage Capacity and Performance” FSx for Windows Specifications+------------------------------------------------------------------+| || Storage Capacity || +----------------------------------------------------------+ || | | || | Minimum: 32 GB | || | Maximum: 65,536 GB (64 TB) | || | Provisioned: SSD or HDD | || | | || +----------------------------------------------------------+ || || Throughput Capacity || +----------------------------------------------------------+ || | | || | SSD: 8 MB/s per TB (up to 12,000 MB/s) | || | HDD: 12 MB/s per TB (up to 1,000 MB/s) | || | | || +----------------------------------------------------------+ || || IOPS || +----------------------------------------------------------+ || | | || | SSD: Up to 500,000 IOPS | || | HDD: Lower IOPS, optimized for throughput | || | | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+19.3 FSx for Lustre
Section titled “19.3 FSx for Lustre”Architecture
Section titled “Architecture” FSx for Lustre+------------------------------------------------------------------+| || Architecture || +----------------------------------------------------------+ || | | || | +----------------------------------------------------+ | || | | Lustre File System | | || | | (High Performance) | | || | +----------------------------------------------------+ | || | | | || | +-------------+-------------+ | || | | | | | || | v v v | || | +----------+ +----------+ +----------+ | || | | Compute | | Compute | | Compute | | || | | Node 1 | | Node 2 | | Node N | | || | +----------+ +----------+ +----------+ | || | | || | Optional: Link to S3 bucket | || | +----------------------------------------------------+ | || | | S3 Bucket (Data Repository) | | || | | - Import data on mount | | || | | - Export changes back to S3 | | || | +----------------------------------------------------+ | || | | || +----------------------------------------------------------+ || || Features: || - POSIX-compliant file system || - High throughput (sub-millisecond latency) || - Parallel file system || - S3 integration || - HPC and ML workloads || |+------------------------------------------------------------------+Deployment Types
Section titled “Deployment Types” FSx for Lustre Deployment Types+------------------------------------------------------------------+| || Scratch || +----------------------------------------------------------+ || | | || | Use Case: | || | - Temporary data | || | - Short-term processing | || | - No data persistence needed | || | | || | Characteristics: | || | - Lower cost | || | - No replication | || | - Data lost if file system fails | || | | || | Types: | || | - Scratch 1 (no HA) | || | - Scratch 2 (higher throughput) | || | | || +----------------------------------------------------------+ || || Persistent || +----------------------------------------------------------+ || | | || | Use Case: | || | - Long-term data storage | || | - Production workloads | || | - Data persistence required | || | | || | Characteristics: | || | - Data replication | || | - Automatic recovery | || | - Higher cost | || | | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+Performance
Section titled “Performance” FSx for Lustre Performance+------------------------------------------------------------------+| || Throughput || +----------------------------------------------------------+ || | | || | Per TB: 50, 100, 200 MB/s | || | Maximum: Up to 12,000 MB/s | || | | || +----------------------------------------------------------+ || || IOPS || +----------------------------------------------------------+ || | | || | Up to 500,000 IOPS | || | Sub-millisecond latency | || | | || +----------------------------------------------------------+ || || Storage || +----------------------------------------------------------+ || | | || | Minimum: 1.2 TB | || | Maximum: 80 TB (scratch), 120 TB (persistent) | || | | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+19.4 FSx for NetApp ONTAP
Section titled “19.4 FSx for NetApp ONTAP” FSx for NetApp ONTAP+------------------------------------------------------------------+| || Architecture || +----------------------------------------------------------+ || | | || | Features: | || | - Multi-protocol access (NFS, SMB, iSCSI) | || | - NetApp ONTAP features | || | - SnapMirror replication | || | - FlexClone | || | - Storage efficiency (deduplication, compression) | || | | || | Use Cases: | || | - Enterprise applications | || | - Database workloads | || | - Mixed protocol environments | || | | || +----------------------------------------------------------+ || || Specifications || +----------------------------------------------------------+ || | | || | Storage: | || | - SSD: Up to 192 TB | || | - Capacity pool: Up to 4 PB | || | | || | Performance: | || | - Up to 12,000 MB/s throughput | || | - Up to 160,000 IOPS | || | | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+19.5 FSx for OpenZFS
Section titled “19.5 FSx for OpenZFS” FSx for OpenZFS+------------------------------------------------------------------+| || Architecture || +----------------------------------------------------------+ || | | || | Features: | || | - ZFS file system | || | - NFS protocol | || | - Snapshots and clones | || | - Compression | || | - Data integrity | || | | || | Use Cases: | || | - Linux workloads | || | - Container storage | || | - Development environments | || | | || +----------------------------------------------------------+ || || Specifications || +----------------------------------------------------------+ || | | || | Storage: | || | - Up to 512 TB | || | | || | Performance: | || | - Up to 1,000,000 IOPS | || | - Sub-millisecond latency | || | | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+19.6 Practical Configuration
Section titled “19.6 Practical Configuration”FSx for Windows with Terraform
Section titled “FSx for Windows with Terraform”# ============================================================# FSx for Windows File Server# ============================================================
resource "aws_fsx_windows_file_system" "main" { storage_capacity = 500 # GB subnet_ids = [var.subnet_id] security_group_ids = [aws_security_group.fsx.id] throughput_capacity = 32 # MB/s
# Active Directory active_directory_id = aws_directory_service_directory.main.id
# Self-managed Active Directory (alternative) # self_managed_active_directory { # dns_ip_addresses = ["10.0.1.10", "10.0.2.10"] # domain_name = "corp.example.com" # password = var.ad_password # username = "Admin" # }
# Deployment type deployment_type = "MULTI_AZ_1"
# Preferred subnet for Multi-AZ preferred_subnet_id = var.subnet_id
# Storage type storage_type = "SSD"
# Automatic backups automatic_backup_retention_days = 30 daily_automatic_backup_start_time = "02:00"
# Tags tags = { Name = "windows-fsx" }}
# ============================================================# FSx for Lustre# ============================================================
resource "aws_fsx_lustre_file_system" "hpc" { storage_capacity = 1200 # GB subnet_ids = [var.subnet_id] security_group_ids = [aws_security_group.fsx.id]
# Deployment type deployment_type = "PERSISTENT_2"
# Throughput per TB per_unit_storage_throughput = 200 # MB/s per TB
# S3 integration import_path = "s3://${aws_s3_bucket.data.id}/input" export_path = "s3://${aws_s3_bucket.data.id}/output"
# Import/Export configuration import_policy { auto_import_policy = "NEW_CHANGED" } export_policy { auto_export_policy = "NEW_CHANGED" }
# Tags tags = { Name = "hpc-fsx" }}
# ============================================================# FSx for NetApp ONTAP# ============================================================
resource "aws_fsx_ontap_file_system" "main" { subnet_ids = var.subnet_ids preferred_subnet_id = var.subnet_ids[0] throughput_capacity = 128 # MB/s
# Tags tags = { Name = "ontap-fsx" }}
resource "aws_fsx_ontap_storage_virtual_machine" "main" { file_system_id = aws_fsx_ontap_file_system.main.id name = "svm-main"
# Active Directory configuration active_directory_configuration { netbios_name = "FSXONTAP" self_managed_active_directory_configuration { dns_ip_addresses = ["10.0.1.10"] domain_name = "corp.example.com" password = var.ad_password username = "Admin" } }}
resource "aws_fsx_ontap_volume" "data" { name = "vol-data" junction_path = "/data" storage_virtual_machine_id = aws_fsx_ontap_storage_virtual_machine.main.id storage_capacity_size = 1024 # GB
# Tiering policy tiering_policy { name = "AUTO" cooling_period = 31 }}
# ============================================================# FSx for OpenZFS# ============================================================
resource "aws_fsx_openzfs_file_system" "main" { subnet_ids = [var.subnet_id] throughput_capacity = 64 # MB/s
# Storage capacity storage_capacity = 1024 # GB
# Tags tags = { Name = "openzfs-fsx" }}
resource "aws_fsx_openzfs_volume" "data" { name = "vol-data" parent_volume_id = aws_fsx_openzfs_file_system.main.root_volume_id
# Storage storage_capacity_reservation_gib = 100 storage_capacity_quota_gib = 500
# Data compression data_compression_type = "ZSTD"}
# ============================================================# Security Group for FSx# ============================================================
resource "aws_security_group" "fsx" { name = "fsx-sg" description = "Security group for FSx" vpc_id = var.vpc_id
# SMB (Windows) ingress { from_port = 445 to_port = 445 protocol = "tcp" cidr_blocks = [var.vpc_cidr] }
# NFS (Lustre, ONTAP, OpenZFS) ingress { from_port = 2049 to_port = 2049 protocol = "tcp" cidr_blocks = [var.vpc_cidr] }
# iSCSI (ONTAP) ingress { from_port = 3260 to_port = 3260 protocol = "tcp" cidr_blocks = [var.vpc_cidr] }
egress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] }
tags = { Name = "fsx-sg" }}19.7 FSx Comparison
Section titled “19.7 FSx Comparison” FSx Service Comparison+------------------------------------------------------------------+| || Feature | Windows | Lustre | ONTAP || ---------------|---------------|---------------|--------------|| Protocol | SMB | POSIX/NFS | NFS/SMB/iSCSI|| Max Size | 64 TB | 120 TB | 4 PB || Max IOPS | 500,000 | 500,000 | 160,000 || Max Throughput | 12,000 MB/s | 12,000 MB/s | 12,000 MB/s || AD Integration | Yes | No | Yes || S3 Integration | No | Yes | No || ---------------|---------------|---------------|--------------|| Use Case | Windows apps | HPC/ML | Enterprise || | Home dirs | Analytics | Databases || | SharePoint | Genomics | Mixed env || |+------------------------------------------------------------------+19.8 Why This Matters in DevOps/SRE
Section titled “19.8 Why This Matters in DevOps/SRE”FSx is critical when you need managed high-performance file systems without the operational burden of self-managed storage clusters. SREs choose FSx for HPC pipelines (Lustre), Windows migrations (FSx Windows), and multi-protocol enterprise storage (ONTAP).
19.9 Linux Systems Perspective
Section titled “19.9 Linux Systems Perspective”FSx Management from Arch Linux
Section titled “FSx Management from Arch Linux”# Install toolssudo pacman -S aws-cli-v2 jq lustre-utils nfs-utils
# FSx status monitoring#!/bin/bash# ~/bin/fsx-status.shecho "=== FSx File Systems ==="aws fsx describe-file-systems \ --query 'FileSystems[*].{ID:FileSystemId,Type:FileSystemType,Status:Lifecycle,Storage:StorageCapacity,Throughput:WindowsConfiguration.ThroughputCapacity||LustreConfiguration.PerUnitStorageThroughput}' \ --output table
# Mount FSx for Lustre on Arch Linuxsudo mkdir -p /mnt/fsxsudo mount -t lustre fs-12345678.fsx.us-east-1.amazonaws.com@tcp:/fsx /mnt/fsx
# Mount FSx for ONTAP (NFS)sudo mount -t nfs -o nfsvers=4.1 svm-12345678.fs-12345678.fsx.us-east-1.amazonaws.com:/vol1 /mnt/ontap
# Check Lustre filesystem statslfs df -h /mnt/fsxlctl get_param llite.*.stats19.10 Troubleshooting Guide
Section titled “19.10 Troubleshooting Guide”| Issue | Cause | Solution |
|---|---|---|
| Can’t mount Lustre | Lustre client not installed | Install lustre-utils matching kernel version |
| FSx Windows slow | Throughput capacity too low | Increase throughput capacity (can modify online) |
| S3 data not visible in Lustre | Data repository association not configured | Create data repository association with import policy |
| ONTAP volume full | Tiering not configured | Enable AUTO tiering policy to capacity pool |
| Multi-AZ failover | Primary node failure | Automatic — verify DNS TTL is low for fast failover |
19.11 Interview Questions
Section titled “19.11 Interview Questions”-
Q: When would you use FSx for Lustre vs EFS?
- A: Lustre: HPC, ML training, genomics — sub-millisecond latency, hundreds of GB/s throughput, S3 data repository integration. EFS: general-purpose NFS, web serving, content management, Lambda integration. Lustre is for compute-intensive parallel workloads; EFS is for shared file storage across services.
-
Q: How does FSx for Lustre integrate with S3?
- A: Lustre creates a data repository association with S3. On mount, file metadata is loaded (lazy loading — data fetched on first access). Changes can auto-export back to S3 via
NEW_CHANGEDpolicy. This enables a pattern: process S3 data at high speed via Lustre, results written back to S3.
- A: Lustre creates a data repository association with S3. On mount, file metadata is loaded (lazy loading — data fetched on first access). Changes can auto-export back to S3 via
19.12 Exam Tips
Section titled “19.12 Exam Tips”
- FSx for Windows: SMB, Active Directory, Windows workloads
- FSx for Lustre: HPC, ML, POSIX, S3 integration
- FSx for ONTAP: Multi-protocol (NFS, SMB, iSCSI), NetApp features
- FSx for OpenZFS: ZFS features, NFS, Linux workloads
- Deployment Types: Single-AZ vs Multi-AZ (Windows), Scratch vs Persistent (Lustre)
- S3 Integration: Lustre can import/export from S3
- Active Directory: Required for Windows, optional for ONTAP
- Performance: Up to 500,000 IOPS for Windows/Lustre
- Storage Capacity: Up to 64 TB (Windows), 120 TB (Lustre), 4 PB (ONTAP)
- Security Groups: Required for all FSx file systems
Next Chapter
Section titled “Next Chapter”Chapter 20: AWS Storage Gateway - Hybrid Storage
Last Updated: March 2026