Databases

The cheat table: Azure data services to AWS

RDS engines & high availability

RDS is the umbrella service for managed relational databases. You pick an engine, a size, a storage type, and AWS handles the rest of the housekeeping.

Multi-AZ vs read replicas - two different things

Azure SQL hides these behind one toggle. RDS exposes them as two distinct features, and you'll pick one or both depending on what you actually want.

Create an RDS Postgres instance

az postgres flexible-server create \
  --resource-group rg-data \
  --name compass-pg \
  --location eastus \
  --tier GeneralPurpose \
  --sku-name Standard_D2ds_v5 \
  --storage-size 64 \
  --version 16 \
  --high-availability ZoneRedundant \
  --admin-user pgadmin \
  --admin-password ""

aws rds create-db-instance \
  --db-instance-identifier compass-pg \
  --db-instance-class db.m6g.large \
  --engine postgres \
  --engine-version 16.3 \
  --allocated-storage 64 \
  --storage-type gp3 \
  --master-username pgadmin \
  --master-user-password "" \
  --multi-az \
  --vpc-security-group-ids sg-0123abc \
  --db-subnet-group-name compass-db-subnets

Same database in Terraform

resource "aws_db_subnet_group" "compass" {
  name       = "compass-db-subnets"
  subnet_ids = [for s in aws_subnet.private : s.id]
}

resource "aws_db_instance" "pg" {
  identifier              = "compass-pg"
  engine                  = "postgres"
  engine_version          = "16.3"
  instance_class          = "db.m6g.large"
  allocated_storage       = 64
  storage_type            = "gp3"
  username                = "pgadmin"
  password                = var.db_password    # move to Secrets Manager in chapter 8
  multi_az                = true
  db_subnet_group_name    = aws_db_subnet_group.compass.name
  vpc_security_group_ids  = [aws_security_group.db.id]
  backup_retention_period = 14
  deletion_protection     = true
  skip_final_snapshot     = false
  final_snapshot_identifier = "compass-pg-final"
  tags = { Project = "Compass" }
}

Aurora - AWS's cloud-native RDBMS

AWS launched Aurora in 2014 with two claims: 5x MySQL performance and 3x PostgreSQL performance for typical OLTP workloads. The marketing numbers come with caveats, but the architecture is genuinely different and the wins on read-heavy workloads are real.

Aurora Serverless v2 - elastic compute, no auto-pause

Aurora Serverless v2 lets you specify a min/max range of ACUs (Aurora Capacity Units) and the cluster scales compute up and down by half-ACU increments in response to load. Unlike v1 (which auto-paused to zero), v2 by default does not pause - the floor is whatever min you set, and you pay for that floor 24x7.

When NOT to pick Aurora

DynamoDB fundamentals - partition keys & items

DynamoDB stores items (records with up to ~400 KB of attributes) in tables. Every table requires a primary key made of:

Create a table

az cosmosdb create \
  --name compass-cosmos \
  --resource-group rg-data \
  --locations regionName=eastus

az cosmosdb sql container create \
  --account-name compass-cosmos \
  --database-name reports \
  --name aggregates \
  --partition-key-path "/report_id" \
  --throughput 400

aws dynamodb create-table \
  --table-name compass-reports-aggregates \
  --attribute-definitions \
      AttributeName=report_id,AttributeType=S \
      AttributeName=period_start,AttributeType=S \
  --key-schema \
      AttributeName=report_id,KeyType=HASH \
      AttributeName=period_start,KeyType=RANGE \
  --billing-mode PAY_PER_REQUEST \
  --tags Key=Project,Value=Compass

resource "aws_dynamodb_table" "events" {
  name         = "global-events"
  billing_mode = "PROVISIONED"
  read_capacity  = 5000
  write_capacity = 5000
  hash_key  = "country_code"   # PK
  range_key = "event_ts"       # SK
  attribute { name = "country_code" ; type = "S" }
  attribute { name = "event_ts"     ; type = "S" }
}

DynamoDB capacity modes - on-demand vs provisioned

The single biggest cost lever in DynamoDB is the billing mode. Pick wrong and you'll over- or under-pay by an order of magnitude.

The crossover - when to switch

Secondary indexes - GSI & LSI

DynamoDB has two flavors of secondary index. Pick wrong and you'll either pay double or fail to add the index at all (LSI must be defined at table-create time).

Projection types - what columns to keep in the index

Define a GSI in Terraform

resource "aws_dynamodb_table" "orders" {
  name         = "orders"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "merchant_id"     # base PK - high cardinality
  range_key    = "order_id"        # base SK

  attribute { name = "merchant_id" ; type = "S" }
  attribute { name = "order_id"    ; type = "S" }
  attribute { name = "status"      ; type = "S" }
  attribute { name = "updated_ts"  ; type = "S" }

  global_secondary_index {
    name            = "status-updated_ts-index"
    hash_key        = "status"        # GSI PK - ok here because
    range_key       = "updated_ts"    # this index is scanned offline only
    projection_type = "KEYS_ONLY"     # worker just needs IDs, looks up rest
  }

  point_in_time_recovery { enabled = true }
  server_side_encryption { enabled = true }
}

The trick here is making the base table partition cleanly by merchant_id (so the live write path is hot-partition-free), and exposing a status-based GSI just for the offline worker that periodically scans for new PENDING items. The GSI's PK can be low-cardinality because nothing is hammering it in real-time.

ElastiCache - Redis and Memcached

ElastiCache has two engines, and the choice is mostly settled by the data structures you need.

Cluster mode vs replica-only

az redis create \
  --name compass-cache \
  --resource-group rg-data \
  --location eastus \
  --sku Standard \
  --vm-size C1 \
  --enable-non-ssl-port false

aws elasticache create-cache-cluster \
  --cache-cluster-id compass-cache \
  --engine valkey \
  --cache-node-type cache.t4g.small \
  --num-cache-nodes 1 \
  --engine-version 7.2 \
  --cache-subnet-group-name compass-cache-subnets \
  --security-group-ids sg-cache0123

Try it: inspect AWS database services

aws dynamodb list-tables --profile compass
# empty {"TableNames": []} on a fresh account is fine

aws dynamodb describe-limits --profile compass

# Returns per-account and per-table provisioned-capacity caps.
# Defaults: 80,000 RCU and 80,000 WCU per region, raisable via support.

aws rds describe-db-engine-versions \
  --engine postgres \
  --query 'DBEngineVersions[?contains(EngineVersion, `16`)].[EngineVersion,Status]' \
  --output table \
  --profile compass

# Look for "available" status; "deprecated" engines still work but won't
# get new features and have an end-of-support date.

aws dynamodb create-table \
  --table-name learn-aws-lab \
  --attribute-definitions AttributeName=id,AttributeType=S \
  --key-schema AttributeName=id,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST \
  --profile compass

aws dynamodb wait table-exists --table-name learn-aws-lab --profile compass
aws dynamodb describe-table --table-name learn-aws-lab --profile compass \
  --query 'Table.[TableName,TableStatus,BillingModeSummary.BillingMode]' \
  --output table

aws dynamodb delete-table --table-name learn-aws-lab --profile compass

Quick check

Gotchas for Azure devs

Project Compass: gfn-reports-aggregates table

Picking up from chapter 5, where we added an S3 bucket for raw telemetry and extended the role's policy. In this chapter's slice we create the DynamoDB table that gfn-reports will write aggregated session metrics into, then attach read/write permissions on the table to gfn-reports-role.

aws dynamodb create-table \
  --table-name gfn-reports-aggregates \
  --attribute-definitions \
      AttributeName=report_id,AttributeType=S \
      AttributeName=period_start,AttributeType=S \
  --key-schema \
      AttributeName=report_id,KeyType=HASH \
      AttributeName=period_start,KeyType=RANGE \
  --billing-mode PAY_PER_REQUEST \
  --sse-specification Enabled=true \
  --tags Key=Project,Value=Compass Key=ManagedBy,Value=learn-aws \
  --profile compass

aws dynamodb wait table-exists \
  --table-name gfn-reports-aggregates \
  --profile compass

aws dynamodb describe-table \
  --table-name gfn-reports-aggregates \
  --query 'Table.[TableName,TableStatus,KeySchema,BillingModeSummary.BillingMode]' \
  --output table \
  --profile compass

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "AggregatesTableRW",
    "Effect": "Allow",
    "Action": [
      "dynamodb:GetItem",
      "dynamodb:BatchGetItem",
      "dynamodb:Query",
      "dynamodb:PutItem",
      "dynamodb:UpdateItem",
      "dynamodb:DeleteItem",
      "dynamodb:BatchWriteItem"
    ],
    "Resource": [
      "arn:aws:dynamodb:us-east-1:<ACCOUNT>:table/gfn-reports-aggregates",
      "arn:aws:dynamodb:us-east-1:<ACCOUNT>:table/gfn-reports-aggregates/index/*"
    ]
  }]
}

aws iam put-role-policy \
  --role-name gfn-reports-role \
  --policy-name AggregatesTableRW \
  --policy-document file:///tmp/compass-dynamo-policy.json \
  --profile compass

# Verify - should now list S3 (from ch5), Lambda basic exec (from ch2), and the new policy
aws iam list-role-policies \
  --role-name gfn-reports-role \
  --profile compass

aws iam list-attached-role-policies \
  --role-name gfn-reports-role \
  --profile compass

Recap & next

What stuck?

• RDS is the relational umbrella - six engines, Multi-AZ for HA, read replicas for scale. They're independent features and you'll often want both.
• Aurora separates storage from compute. 15 read replicas, fast failover, shared storage fleet across 3 AZs. Pick it for Postgres/MySQL workloads unless cost or version constraints push you to vanilla RDS.
• Aurora Serverless v2 scales compute by 0.5 ACU but doesn't pause to zero by default. Cheaper than always-on Aurora for spiky loads, more expensive than Azure SQL Serverless for truly idle workloads.
• DynamoDB partition keys are the single most important design choice. Low cardinality = hot partition = throttling that no amount of provisioned capacity will fix.
• On-demand vs provisioned: on-demand is the right default; provisioned is right once the workload is predictable. Crossover ~1,000 sustained req/s.
• GSIs are separate replicated indexes with their own capacity, always eventually consistent. LSIs share the base table's partition and support strong reads but must be defined at creation.
• ElastiCache = managed Redis (Valkey, post-2024) or Memcached. VPC-only, no public endpoint. Cluster mode for horizontal scale.

The mental model in one sentence

Common pitfalls so far