Storage

The cheat table: Azure Storage → AWS storage

S3 fundamentals: buckets, keys, regions

Three properties of S3 catch every Azure dev off guard:

Make a bucket: CLI side-by-side

# 1. Create the storage account (parent)
az storage account create \
  --name mystgacct \
  --resource-group rg-data \
  --location eastus \
  --sku Standard_LRS

# 2. Create a blob container inside it
az storage container create \
  --account-name mystgacct \
  --name reports

# 3. Upload
az storage blob upload \
  --account-name mystgacct \
  --container-name reports \
  --name report.pdf \
  --file ./report.pdf

# 1. Create the bucket (no parent to create first)
aws s3 mb s3://my-org-reports-2025-05 \
  --region us-east-1

# 2. Upload
aws s3 cp ./report.pdf \
  s3://my-org-reports-2025-05/reports/report.pdf

# 3. List
aws s3 ls s3://my-org-reports-2025-05/reports/

# Notice: no "container" step. The bucket IS the container.
# The 'reports/' prefix is just part of the key string.

Same bucket in Terraform

resource "aws_s3_bucket" "reports" {
  bucket = "my-org-reports-2025-05"

  tags = {
    Project    = "Compass"
    DataClass  = "internal"
    ManagedBy  = "terraform"
  }
}

# Versioning is a SEPARATE resource since the v4 provider split.
# A common bug source - don't forget to attach it.
resource "aws_s3_bucket_versioning" "reports" {
  bucket = aws_s3_bucket.reports.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_public_access_block" "reports" {
  bucket                  = aws_s3_bucket.reports.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

Storage classes & the cost ladder

S3 offers six "live" storage classes plus Reduced Redundancy (legacy, deprecated). Pick the wrong one and your storage bill goes 80% lower or your retrieval bill goes 10x higher. The trick: storage-class is a per-object attribute, not a per-bucket setting, so you can mix.

Prices are us-east-1 list, approximate, accurate to within ~5% as of early 2026. Check the official pricing page before basing a decision on these. Pricing in other regions is higher.

Tier mapping vs Azure

Set a class on upload, or transition later

# Set storage class at upload time
aws s3 cp big-report.parquet s3://my-bucket/2025/q1/big-report.parquet \
  --storage-class STANDARD_IA

# Or for many objects with a recursive copy
aws s3 cp ./archive s3://my-bucket/archive/ \
  --recursive \
  --storage-class GLACIER

# Change class of an existing object (server-side copy, NOT free):
aws s3 cp s3://my-bucket/old.log s3://my-bucket/old.log \
  --storage-class GLACIER_IR \
  --metadata-directive COPY

# Check current class:
aws s3api head-object \
  --bucket my-bucket \
  --key 2025/q1/big-report.parquet \
  --query StorageClass

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "AWS": "arn:aws:iam::444444444444:role/PartnerReader" },
    "Action": ["s3:ListBucket", "s3:GetObject"],
    "Resource": "arn:aws:s3:::gfn-reports-shared"
  }]
}

Lifecycle rules: automate the cooling

Manually downgrading storage class on every object would be madness. Lifecycle rules are JSON declarations on the bucket that say "after N days, transition matching objects to class X; after M days, expire them entirely." Same idea as Azure's lifecycle management rules, slightly different schema.

The JSON, side-by-side with Azure

{
  "rules": [{
    "name": "tier-and-expire",
    "enabled": true,
    "type": "Lifecycle",
    "definition": {
      "filters": {
        "blobTypes": ["blockBlob"],
        "prefixMatch": ["reports/"]
      },
      "actions": {
        "baseBlob": {
          "tierToCool":    { "daysAfterModificationGreaterThan": 30 },
          "tierToArchive": { "daysAfterModificationGreaterThan": 90 },
          "delete":        { "daysAfterModificationGreaterThan": 365 }
        }
      }
    }
  }]
}

{
  "Rules": [{
    "ID": "tier-and-expire",
    "Status": "Enabled",
    "Filter": {
      "Prefix": "reports/"
    },
    "Transitions": [
      {
        "Days": 30,
        "StorageClass": "STANDARD_IA"
      },
      {
        "Days": 90,
        "StorageClass": "GLACIER"
      }
    ],
    "Expiration": {
      "Days": 365
    }
  }]
}

Apply the lifecycle policy

# Save the rule to a file (see JSON above), then:
aws s3api put-bucket-lifecycle-configuration \
  --bucket gfn-reports-raw \
  --lifecycle-configuration file://lifecycle.json

# Verify
aws s3api get-bucket-lifecycle-configuration --bucket gfn-reports-raw

# Remove all rules (single call removes the whole config):
aws s3api delete-bucket-lifecycle --bucket gfn-reports-raw

Versioning + MFA delete: in-place edits are a lie

Versioning is opt-in (per bucket), and once turned on it's irreversible: you can only suspend it, not "go back to never having had it". The two states most teams ship with:

Turn it on, look around, roll back

# Enable versioning on a bucket
aws s3api put-bucket-versioning \
  --bucket gfn-reports-raw \
  --versioning-configuration Status=Enabled

# Check current status
aws s3api get-bucket-versioning --bucket gfn-reports-raw

# Upload twice to create two versions
echo "v1" | aws s3 cp - s3://gfn-reports-raw/notes.txt
echo "v2" | aws s3 cp - s3://gfn-reports-raw/notes.txt

# List all versions of a key
aws s3api list-object-versions \
  --bucket gfn-reports-raw \
  --prefix notes.txt

# Restore an older version by copying it over the current one
aws s3api copy-object \
  --bucket gfn-reports-raw \
  --copy-source "gfn-reports-raw/notes.txt?versionId=ABC123..." \
  --key notes.txt

# Delete a specific version (the delete-version op, NOT a delete marker)
aws s3api delete-object \
  --bucket gfn-reports-raw \
  --key notes.txt \
  --version-id ABC123...

MFA delete: don't enable unless you mean it

MFA delete is a bucket-level setting requiring an MFA token to either permanently delete a version or to change the versioning state. It can only be enabled by the AWS account's root user via the API (not the console, not IAM users) and the same root user is required to disable it. Most teams don't enable it because it makes Terraform-driven bucket destruction painful.

EBS volume types: block storage for EC2

EC2 instance storage comes in two flavors: instance store (ephemeral, on the host, free, dies with the instance) and EBS (network-attached, durable, billed per GB-month). 95% of workloads want EBS for anything important.

Creating a volume side-by-side

az disk create \
  --resource-group rg-app \
  --name data-disk \
  --size-gb 100 \
  --sku Premium_LRS

az vm disk attach \
  --resource-group rg-app \
  --vm-name myvm \
  --name data-disk

aws ec2 create-volume \
  --availability-zone us-east-1a \
  --size 100 \
  --volume-type gp3 \
  --iops 3000 \
  --throughput 125 \
  --tag-specifications "ResourceType=volume,Tags=[{Key=Name,Value=data-disk}]"

aws ec2 attach-volume \
  --volume-id vol-abc123 \
  --instance-id i-xyz789 \
  --device /dev/sdf

EFS & the FSx family: shared filesystems

When you need POSIX semantics and multiple compute nodes hitting the same files, you reach for EFS (Linux/NFS) or one of the FSx variants. Each has a niche:

Mount EFS from EC2 or EKS

# Install the helper (Amazon Linux: amazon-efs-utils)
sudo dnf install -y amazon-efs-utils

# Mount using the EFS mount helper - handles TLS in transit
sudo mkdir /mnt/shared
sudo mount -t efs -o tls fs-0abc123:/ /mnt/shared

# Test - any other instance mounting the same fs-id sees these files
echo "hello from $(hostname)" | sudo tee /mnt/shared/who.txt
cat /mnt/shared/who.txt

resource "aws_efs_file_system" "shared" {
  creation_token   = "gfn-shared-fs"
  performance_mode = "generalPurpose"
  throughput_mode  = "elastic"

  lifecycle_policy {
    transition_to_ia = "AFTER_30_DAYS"
  }

  tags = { Project = "Compass" }
}

# One mount target per AZ you want to access from
resource "aws_efs_mount_target" "a" {
  file_system_id  = aws_efs_file_system.shared.id
  subnet_id       = aws_subnet.private_a.id
  security_groups = [aws_security_group.efs.id]
}

Try it: make a bucket, version it, kill it

export BUCKET="learn-aws-${USER}-$(date +%Y%m%d)"
echo "$BUCKET"

aws s3 mb "s3://$BUCKET" --region us-east-1

aws s3api put-bucket-versioning \
  --bucket "$BUCKET" \
  --versioning-configuration Status=Enabled

aws s3api get-bucket-versioning --bucket "$BUCKET"
# Output: { "Status": "Enabled" }

echo "version one" > /tmp/lab.txt
aws s3 cp /tmp/lab.txt "s3://$BUCKET/lab.txt"

echo "version two - overwriting" > /tmp/lab.txt
aws s3 cp /tmp/lab.txt "s3://$BUCKET/lab.txt"

aws s3api list-object-versions --bucket "$BUCKET"
# Two distinct versions, each with their own VersionId. The "IsLatest: true" one is the second.

aws s3 rb "s3://$BUCKET" --force

# Verify it's gone (should 404):
aws s3api head-bucket --bucket "$BUCKET"

Quick check

Gotchas for Azure devs

Project Compass: telemetry bucket

The Compass service needs a place to dump raw GeForce NOW session telemetry before the Lambda we deployed in chapter 4 processes it. In this slice we create that bucket, turn on versioning (so a misbehaving producer can't silently corrupt history), set a lifecycle policy that keeps recent data hot and old data cheap, and finally extend gfn-reports-role with the bucket-specific S3 permissions it needs.

# Bucket name must be globally unique - add account ID for safety.
ACCT=$(aws sts get-caller-identity --query Account --output text --profile compass)
BUCKET="gfn-reports-raw-${ACCT}"

aws s3 mb "s3://${BUCKET}" \
  --region us-east-1 \
  --profile compass

# Block all public access (defense-in-depth)
aws s3api put-public-access-block \
  --bucket "${BUCKET}" \
  --public-access-block-configuration \
    BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true \
  --profile compass

# Enable versioning
aws s3api put-bucket-versioning \
  --bucket "${BUCKET}" \
  --versioning-configuration Status=Enabled \
  --profile compass

# Tag it so we can find / clean up later
aws s3api put-bucket-tagging \
  --bucket "${BUCKET}" \
  --tagging 'TagSet=[{Key=Project,Value=Compass},{Key=ManagedBy,Value=learn-aws}]' \
  --profile compass

{
  "Rules": [{
    "ID": "telemetry-tiering",
    "Status": "Enabled",
    "Filter": { "Prefix": "" },
    "Transitions": [
      { "Days": 30,  "StorageClass": "STANDARD_IA" },
      { "Days": 90,  "StorageClass": "GLACIER" }
    ],
    "Expiration": { "Days": 365 },
    "NoncurrentVersionExpiration": { "NoncurrentDays": 30 }
  }]
}

aws s3api put-bucket-lifecycle-configuration \
  --bucket "${BUCKET}" \
  --lifecycle-configuration file:///tmp/compass-lifecycle.json \
  --profile compass

# Verify
aws s3api get-bucket-lifecycle-configuration \
  --bucket "${BUCKET}" \
  --profile compass

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BucketLevel",
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": "arn:aws:s3:::gfn-reports-raw-*"
    },
    {
      "Sid": "ObjectLevel",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:GetObjectVersion"
      ],
      "Resource": "arn:aws:s3:::gfn-reports-raw-*/*"
    }
  ]
}

aws iam put-role-policy \
  --role-name gfn-reports-role \
  --policy-name compass-s3-access \
  --policy-document file:///tmp/compass-s3-policy.json \
  --profile compass

# Verify
aws iam list-role-policies --role-name gfn-reports-role --profile compass
aws iam get-role-policy \
  --role-name gfn-reports-role \
  --policy-name compass-s3-access \
  --profile compass

Recap & next

What stuck?

• S3 is global-name, region-pinned, flat-key. Folders are an illusion; buckets are unique across all of AWS; objects are (key, body, metadata) triples up to 5 TB.
• Six storage classes: Standard / Standard-IA / One Zone-IA / Glacier Instant Retrieval / Glacier Flexible / Glacier Deep Archive, plus Intelligent-Tiering. Cheaper storage = pricier retrievals + minimum-duration penalties.
• Lifecycle rules automate transitions and expiration but cost per-object transition fees - lethal for billions of tiny files.
• Versioning turns PUT-overwrites into immutable history. Once on, only suspend, not undo. Lifecycle the noncurrent versions or storage piles up.
• EBS = network-attached block storage for EC2. gp3 is the new default; io2 for IOPS-critical; st1/sc1 for sequential/cold workloads.
• EFS = managed NFS for multi-node Linux workloads. The FSx family wraps Windows, Lustre, NetApp ONTAP, and OpenZFS.

The mental model in one sentence

Common pitfalls so far