The final chapter, and the one where real-world AWS thinking actually starts. One account isn't enough; clickops doesn't scale; CloudFormation will appear in your life whether you want it or not - and Terraform earns its keep right here. We end where many teams should have started.
Azure devs are used to a world where one subscription per environment, ARM/Bicep, and Management Groups are the obvious answer. AWS gets there via a different path: AWS Organizations for the account tree, SCPs for hard guardrails, Control Tower for the opinionated starter, and Terraform or CloudFormation for the actual resource shapes. By the end of this chapter you will have reified everything from chapters 2-11 into one Terraform module - and you'll know where to point yourself next.
| Concept | Azure | AWS |
|---|---|---|
| Native declarative IaC | Bicep (modern) / ARM templates (older JSON) | CloudFormation (YAML or JSON) |
| Imperative IaC SDK | Bicep with modules; Pulumi (third party) | AWS CDK - Python / TypeScript / Go / Java that compiles to CloudFormation |
| Multi-cloud IaC | Terraform azurerm provider | Terraform aws provider |
| Account / sub container | Subscription | Account |
| Group of accounts/subs | Management Group | Organizational Unit (OU) inside AWS Organizations |
| Tenant / org root | AAD tenant | AWS Organization (with a management account at the root) |
| Org-level guardrails | Azure Policy at MG / sub scope | SCPs (Service Control Policies) + AWS Config rules |
| Opinionated multi-account starter | Azure Landing Zones (CAF) / Enterprise-scale | AWS Control Tower (builds on Organizations + Config + SSO) |
| Resource grouping API | Azure Resource Manager (ARM) | CloudFormation Stacks (closest equivalent - tracks a set of resources together) |
| Multi-account/region rollout | Blueprints (deprecating) / Policy at scope | CloudFormation StackSets - one template, many target accounts & regions |
| Account creation factory | Subscription Vending solution (custom, ALZ) | Control Tower Account Factory (or via Service Catalog / IaC) |
| Consolidated billing | EA / MCA billing scope | Built into Organizations - one bill, RIs/SP shared across accounts |
| Cross-account IAM read | AAD is shared across subs | IAM is per-account; cross-account via AssumeRole or Identity Center |
| State storage (TF) | Azure Storage backend + blob lease lock | S3 backend + DynamoDB table for locking (the canonical pattern) |
cdk synth time it compiles to a CloudFormation template, which CFN then deploys. So CDK ships CFN under the hood - all CFN limits apply.terraform plan, it reads the notebook, asks AWS what really exists, and tells you the difference. Lose the notebook and Terraform forgets it ever made anything - so it'll try to create everything from scratch and you'll have two copies of everything. The notebook is sacred.If you've used Terraform against Azure, the AWS provider feels almost identical at the surface - HCL syntax, the same resource blocks, the same plan / apply rhythm. What changes is the resource names and what's state-locked behind the scenes.
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.0"
}
}
backend "azurerm" {
resource_group_name = "rg-tfstate"
storage_account_name = "tfstateprod"
container_name = "tfstate"
key = "reports.tfstate"
}
}
provider "azurerm" { features {} }
resource "azurerm_storage_account" "reports" {
name = "reportsprod001"
resource_group_name = "rg-reports"
location = "eastus"
account_tier = "Standard"
account_replication_type = "GRS"
}
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
backend "s3" {
bucket = "compass-tfstate-prod"
key = "reports.tfstate"
region = "us-east-1"
dynamodb_table = "compass-tfstate-lock"
encrypt = true
}
}
provider "aws" { region = "us-east-1" }
resource "aws_s3_bucket" "reports" {
bucket = "compass-reports-prod"
}
resource "aws_s3_bucket_versioning" "reports" {
bucket = aws_s3_bucket.reports.id
versioning_configuration { status = "Enabled" }
}
Two things stand out. First: the backend block. Azure uses one storage account; AWS canonically uses an S3 bucket plus a DynamoDB table. We'll see why next. Second: resource granularity. The AWS provider tends to split things that Azure bundles together - aws_s3_bucket and aws_s3_bucket_versioning are separate resources, where Azure puts replication on the same resource. This isn't worse, just more atomic - and it lets you change one knob without touching the others.
Terraform state is just a JSON file. If two engineers run terraform apply at the same time and write to the same state file, you get corruption - resource IDs overwritten, drift, the works. The fix is a lock: one engineer holds it, the other waits.
terraform {
backend "s3" {
bucket = "compass-tfstate-prod"
key = "envs/prod/reports.tfstate"
region = "us-east-1"
dynamodb_table = "compass-tfstate-lock" # the lock
encrypt = true # SSE-KMS
kms_key_id = "alias/aws/s3" # or your CMK
}
}
A Terraform module is a folder of .tf files that packages a reusable shape. Every Terraform config is itself a module (the root module). Calling another module is a single block.
module reports './modules/reports.bicep' = {
name: 'reports-dep'
params: {
location: 'eastus'
environment: 'prod'
storageReplica: 'GRS'
}
}
module "reports" {
source = "../../modules/gfn-reports"
environment = "prod"
region = "us-east-1"
tags = {
Project = "Compass"
ManagedBy = "terraform"
}
}
For one stack in one account, plain Terraform is plenty. But once you have dev, staging, prod in three accounts, you find yourself copy-pasting backend.tf across folders and praying nobody mistypes a bucket name. Terragrunt generates the backend block from a single terragrunt.hcl at the root and inherits it into each env. It also gives you run-all apply which walks a dependency graph across modules.
include "root" {
path = find_in_parent_folders()
}
inputs = {
environment = "prod"
region = "us-east-1"
}
Terragrunt is optional, opinionated, and adds a layer of indirection. Use it when the boilerplate cost actually hurts; don't use it because someone on the internet said you must.
A teammate writes a small module wrapping an S3 bucket that's conditionally created via the enabled variable. They want the bucket ARN exposed as an output so callers can use it. They run terraform plan with enabled = false and the plan errors out before even rendering. What's wrong?
resource "aws_s3_bucket" "this" {
count = var.enabled ? 1 : 0
bucket = "compass-reports-${var.environment}"
}
output "bucket_arn" {
value = aws_s3_bucket.this.arn # <-- error here
}
count meta-argument turns the resource into a list, even when count is 1.
Terraform error: Because aws_s3_bucket.this has "count" set, its attributes must be accessed on specific instances. The fix when enabled is always intended to be true: aws_s3_bucket.this[0].arn. To safely handle the enabled = false case where the bucket doesn't exist, gate it: try(aws_s3_bucket.this[0].arn, null) or length(aws_s3_bucket.this) > 0 ? aws_s3_bucket.this[0].arn : null.
Same bug shape applies to for_each - the resource becomes a map, indexed by the key. aws_s3_bucket.this["primary"].arn, not aws_s3_bucket.this.arn. The rule: once you add count or for_each, you've changed the access shape of every reference to that resource.
version = "~> 5.0" (or whatever major) in required_providers. The AWS provider ships breaking changes between majors (renamed attributes, removed deprecated args). Without a pin, a fresh terraform init on a coworker's laptop six months from now picks up v6, fails to plan, and you spend an afternoon bisecting.
You don't have to use CloudFormation. But you'll encounter CloudFormation because: many AWS services accept a "deploy via CFN button" pattern (SAM, Quick Starts), CDK compiles to it, and StackSets is the only first-party way to deploy one template across many accounts. Knowing how to read a CFN template is a literacy floor.
AWSTemplateFormatVersion: "2010-09-09"
Description: Compass reports bucket + DynamoDB table
Parameters:
Environment:
Type: String
AllowedValues: [dev, prod]
Resources:
ReportsBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub "compass-reports-${Environment}"
VersioningConfiguration: { Status: Enabled }
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault: { SSEAlgorithm: AES256 }
ReportsTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: !Sub "compass-reports-${Environment}"
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- { AttributeName: pk, AttributeType: S }
KeySchema:
- { AttributeName: pk, KeyType: HASH }
Outputs:
BucketArn:
Value: !GetAtt ReportsBucket.Arn
Export: { Name: !Sub "${AWS::StackName}-BucketArn" }
| Aspect | CloudFormation | Terraform (aws provider) |
|---|---|---|
| Cost | Free (you pay only for resources) | Free OSS; HCP Terraform/Enterprise is paid; state storage is a few cents |
| State | Managed by AWS, invisible to you | Your problem - S3 + DynamoDB pattern |
| New service coverage | Often late by months | Often fastest, including pre-GA APIs via beta providers |
| Drift detection | Native, but explicit (aws cloudformation detect-stack-drift) | Implicit on every plan; cleaner UX |
| Multi-cloud | AWS only | Yes - one tool, many providers |
| Multi-account/region rollout | StackSets - first-class, native, free | Terraform workspaces + provider aliases, or third-party (Spacelift, env0) |
| Failure recovery | Auto-rollback (sometimes painfully so) | Errors halt; you fix and re-run |
| Modularity | Nested stacks (clunky), or CDK constructs | Modules - cleaner, more reusable |
| Day-2 ops | "Tear it down and recreate" can be hostile | Surgical terraform state commands |
This is where CloudFormation pulls ahead of Terraform for true multi-account work. A StackSet is a template plus a list of target accounts and regions. You deploy it once from the management account; AWS rolls the underlying stacks out everywhere. New account joins the OU? Auto-deployed via "trusted access" - no extra effort.
cdk synth, your code outputs a giant CloudFormation YAML file. cdk deploy hands that file to CloudFormation. So under the hood you're still running CloudFormation; the value-add is that you didn't have to write 800 lines of YAML by hand.import * as cdk from 'aws-cdk-lib';
import * as s3 from 'aws-cdk-lib/aws-s3';
class ReportsStack extends cdk.Stack {
constructor(scope, id, props) {
super(scope, id, props);
new s3.Bucket(this, 'Reports', {
versioned: true,
encryption: s3.BucketEncryption.S3_MANAGED,
});
}
}
from aws_cdk import Stack, aws_s3 as s3
from constructs import Construct
class ReportsStack(Stack):
def __init__(self, scope: Construct, cid: str, **kw):
super().__init__(scope, cid, **kw)
s3.Bucket(self, "Reports",
versioned=True,
encryption=s3.BucketEncryption.S3_MANAGED)
Your team is already deep in a single language (TS / Python) and wants infra in the same repo and review process as app code. You like high-level "L2 constructs" that bundle sensible defaults (a Bucket comes with versioning, encryption, public-access blocks).
You operate across multiple clouds, you want a deterministic plan, you want a thriving public module ecosystem, or you've been burned by CloudFormation's rollback-into-UPDATE_ROLLBACK_FAILED hell.
You want zero new tooling, you have a one-off "deploy this from the AWS docs" snippet, or you need StackSets for org-wide fan-out. Also: many AWS Quick Starts only ship as CFN.
continue-update-rollback with a manual list of resources to skip. CDK doesn't escape these - it just gives you nicer syntax on top of them.
"2010-09-09" in every modern template? That's the date AWS finalized the original spec, still pinned in 2026.
An AWS Organization is the answer to "how do I run 12 environments + production + sandbox + audit + log-archive accounts without losing my mind?". You create one Organization, add accounts, group them into OUs, and apply policies at the OU level.
Azure devs often run prod and dev in the same subscription with separate resource groups. In AWS the convention is one account per environment. The reasoning:
| Benefit | What it gives you |
|---|---|
| Hard blast-radius | A typo, an automation bug, or a compromised credential in dev cannot see or touch prod. Account-level isolation is the strongest boundary AWS offers. |
| Per-account quotas | Most service quotas (Lambda concurrent, EC2 vCPU, EIPs) are per-account. Prod won't get throttled because dev ran a load test. |
| Easier billing attribution | "What did prod cost?" is a single account-id filter in Cost Explorer. |
| Cleaner deletion | End-of-life a project? Close the account. Resource cleanup is automatic. |
| Free | Accounts have no monthly fee. The only cost is the resources inside them. |
SCPs are the only AWS mechanism that can stop the root user doing something. They're attached to an OU (or the org root, or one account) and apply to everything in that scope. IAM policies and resource policies must also allow the action - if the SCP denies it, you're done.
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Deny",
"Action": "*",
"Resource": "*",
"Condition": {
"StringNotEquals": {
"aws:RequestedRegion": ["us-east-1", "us-west-2"]
},
"ArnNotLike": {
"aws:PrincipalARN": ["arn:aws:iam::*:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_BreakGlass_*"]
}
}
}]
}
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Deny",
"Action": [
"cloudtrail:StopLogging",
"cloudtrail:DeleteTrail",
"cloudtrail:UpdateTrail"
],
"Resource": "*"
}]
}
Version, same Effect, same Action / Resource / Condition. What changes is the semantic: an Allow statement in an SCP doesn't grant anything; it just permits the IAM evaluation underneath. The default SCP that ships with a new Organization is FullAWSAccess - "everything is allowed by the org" - and IAM does the rest of the work. You only add Deny SCPs.
resource "aws_organizations_policy" "deny_regions" {
name = "deny-non-home-regions"
description = "Block API calls outside us-east-1 and us-west-2"
type = "SERVICE_CONTROL_POLICY"
content = file("${path.module}/scps/deny-leaving-home-region.json")
}
resource "aws_organizations_policy_attachment" "deny_regions_workloads" {
policy_id = aws_organizations_policy.deny_regions.id
target_id = aws_organizations_organizational_unit.workloads.id
}
A platform engineer is cleaning up an old staging environment. They cd into the right folder, type terraform workspace list to confirm... but they're looking at the output from a different tmux pane. The workspace they think is selected is staging; the workspace actually selected is prod. They run terraform destroy -auto-approve.
By the time someone notices the alarms going off, Terraform has torn down 200 production resources: the EKS cluster, three RDS databases, the customer-facing ALB, all the NAT gateways, the KMS keys. The state file - kept locally on the engineer's laptop, no remote backend, no DynamoDB lock - had been corrupted partway through, so the team can't even use it to figure out what existed.
They spend the next four hours reconstructing prod from a six-week-old CloudFormation drift report and frantic aws ec2 describe-* calls. The DB they restore is two hours stale because the most recent snapshot was on the destroyed RDS volume. Total rebuild + lost-engineering time: about $30K. Total customer-visible outage: 4 hours.
Lessons: (1) always use a remote state backend with DynamoDB locking - the bug would have been caught at apply-time, not after. (2) Separate workspaces per env with crystal-clear naming, and prompt for confirmation in your shell (a PS1 that shows the AWS profile + TF workspace is free insurance). (3) Put lifecycle { prevent_destroy = true } on irreplaceable resources - it blocks terraform destroy outright. (4) Treat the production state file with the same paranoia as production data. Versioned S3 bucket, KMS-encrypted, restricted IAM, audit-logged via CloudTrail.
A team is cleaning up a misbehaving Terraform config. The state file has gone out of sync with reality (some resources got renamed in AWS by an emergency clickops change). To "fix" it they run terraform state rm on a handful of resources - an RDS instance, two NAT gateways, an ALB, a few EBS volumes - intending to terraform import them back in a clean way later. They get pulled into a different fire and forget. Next sprint someone runs terraform apply from the same module. A month later the AWS bill is $1,500 higher than expected and nobody can explain why. What's happening?
terraform state rm doesn't delete the AWS resource. It only deletes the record of it from Terraform's notebook.
When terraform apply ran next, Terraform looked at its (now-incomplete) state, saw "I'm supposed to have an RDS, an ALB, two NAT gateways, etc., but my notebook says none of these exist", and dutifully created fresh ones. The original resources kept running. So now there are two of each.
The math: 1 extra db.r6g.large RDS = ~$220/mo. 2 extra NAT gateways = ~$65/mo + data. 1 extra ALB = ~$22/mo. 5 extra gp3 EBS volumes = ~$50/mo. Add CloudWatch metrics, KMS calls, cross-AZ data on the doubled NATs - the bill creeps up by roughly $1,500/month and stays there until someone notices.
Fix: use terraform import to bring the existing resources back into state (this is the real recovery path - never blindly state rm without a planned import to follow). For finding shadow resources right now: filter Cost Explorer by service, then by tag absence (anything missing your ManagedBy = terraform tag is suspicious), then by creation date. AWS Config's "resource inventory" view is invaluable here. Long-term: add a CI check that fails the build if terraform plan shows resources that match production naming but want to be created.
AWS Organizations gives you the tree. Control Tower gives you the opinionated starter content inside the tree - the standard accounts, the standard SCPs, the standard auditing, the standard SSO. A Control Tower-managed Org is called a Landing Zone.
The end state. A well-architected multi-account AWS environment with org structure, baseline SCPs, centralized logging, and identity. You can build one by hand with Organizations + IaC, or you can let Control Tower build it for you.
The tool. An AWS service that provisions and continuously enforces a Landing Zone. It runs the "set up landing zone" workflow, provisions log-archive + audit accounts, and gives you Account Factory for new accounts. Costs nothing extra; the underlying services (Config, CloudTrail, Identity Center) cost their usual amounts.
Security (for log-archive + audit) and Sandbox (for experimentation). You add more OUs (Workloads, Suspended) as needed.No apply, no resources created. The point is to see the workflow end-to-end and to look at the Org you (or your enterprise) already have.
Step 1. Create a tiny module that wants one S3 bucket. Use a name that doesn't exist anywhere on Earth (S3 bucket names are globally unique).
mkdir -p /tmp/tf-lab && cd /tmp/tf-lab
cat > main.tf <<'EOF'
terraform {
required_providers {
aws = { source = "hashicorp/aws", version = "~> 5.0" }
}
}
provider "aws" {
region = "us-east-1"
profile = "compass"
}
resource "aws_s3_bucket" "demo" {
bucket = "compass-tf-lab-${random_id.s.hex}"
}
resource "random_id" "s" { byte_length = 4 }
EOF
Step 2. Init and plan. Note: this uses a local backend, which is fine for a throwaway sandbox.
terraform init
# Downloads the AWS provider, sets up .terraform/
terraform plan
# Plan: 2 to add, 0 to change, 0 to destroy.
# No resources will actually be created - this is a dry run.
Step 3. Inspect your AWS Organization. Read-only, no charges.
# Are you in an Organization at all?
aws organizations describe-organization --profile compass
# If yes, list the SCPs attached to your current account:
ACCT=$(aws sts get-caller-identity --query Account --output text --profile compass)
aws organizations list-policies-for-target \
--target-id $ACCT \
--filter SERVICE_CONTROL_POLICY \
--profile compass
# List the OUs at the root of the org:
ROOT=$(aws organizations list-roots --query 'Roots[0].Id' --output text --profile compass)
aws organizations list-organizational-units-for-parent \
--parent-id $ROOT \
--profile compass
Step 4. Tear down the lab (no AWS resources to clean up - we only ran plan).
cd / && rm -rf /tmp/tf-lab
What you learned: Terraform's plan/apply rhythm, the AWS provider's surface, and the read-only Organizations APIs you'll use forever to investigate "why am I being denied". If your org has SCPs attached above you, every "AccessDenied" investigation should start with list-policies-for-target.
The final chapter quiz. These are the questions most likely to bite in real multi-account work.
1. In the canonical Terraform-on-AWS backend, what is the DynamoDB table actually used for?
terraform plan output so re-runs are faster.ConditionExpression that the row must not already exist. If two engineers try to apply concurrently, the second one's write fails the condition, and Terraform prints "Lock held by ...". When the apply finishes, Terraform deletes the row.
2. An SCP at the org root denies ec2:RunInstances. The root user of a member account signs in and tries to launch an EC2 instance via the console. What happens?
3. What's the relationship between Control Tower and a Landing Zone?
4. You need to deploy the same CloudFormation template to 12 AWS accounts in 3 regions each - that's 36 stack instances. The template provisions a CloudWatch log forwarder. Which is the right tool?
aws cloudformation create-stack calls in a shell loop.cdk deploy --all.5. Your team is deciding between CDK and Terraform for a new greenfield AWS-only platform. Which factor most strongly favors CDK?
plan step before any change touches AWS.npm test on both, and you bring your testing instincts to infra. The other options favor Terraform: (a) multi-cloud is Terraform's bread and butter; (b) CDK's cdk diff is fine but Terraform's plan is more deterministic and is a first-class artifact; (d) Terraform's new-service coverage is usually faster than CFN/CDK.
*.tfstate* to .gitignore on day one. If you committed it already, treat it as a leaked-secrets incident: rotate everything in the file.
terraform apply calls will write to the same state object and clobber each other - resource IDs lost, drift introduced, sometimes total state corruption. The fix is one DynamoDB table with LockID as a string hash key (smallest possible table - pennies a month). Add dynamodb_table = "..." to your backend block. Never skip this. If you're using S3 native conditional writes (a newer alternative on the latest TF versions), confirm with your team that it's actually enabled - the DynamoDB pattern is still the safest default.
terraform destroy with the wrong context will happily destroy production. Terraform doesn't ask "are you sure this is staging?" - it asks "are you sure?" and you say yes. Defenses: shell prompt that shows AWS profile + TF workspace, lifecycle { prevent_destroy = true } on irreplaceable resources, separate accounts per env (so even if you point at the wrong one, an SCP can deny the destroy), and CI-driven applies so humans rarely run destroy manually.
This is the final slice. Across chapters 1-11 you built gfn-reports piece by piece: a profile, an IAM role, a VPC, a Lambda, an S3 bucket, a DynamoDB table, an SQS queue, KMS keys + Secrets Manager, CloudWatch alarms, an EKS worker, and an API Gateway. All of that was clickops or one-off CLI commands. Now we pack it into one Terraform module.
Why now: you've seen every piece individually. Putting them in one module turns "I clicked some buttons" into "we have a reproducible, code-reviewed, multi-environment artifact". This is what teams ship to prod.
compass-infra/
├── bootstrap/ # local state, runs once
│ └── main.tf # creates the S3 bucket + DDB lock table
├── modules/
│ └── gfn-reports/
│ ├── main.tf # the whole stack
│ ├── variables.tf
│ ├── outputs.tf
│ └── README.md
├── envs/
│ ├── dev/
│ │ ├── backend.tf # S3 backend, dev key
│ │ ├── main.tf # calls module with dev inputs
│ │ └── terraform.tfvars
│ └── prod/
│ ├── backend.tf # S3 backend, prod key
│ ├── main.tf # calls module with prod inputs
│ └── terraform.tfvars
└── org/ # Organization-level (separate state)
├── ous.tf
└── scps.tf
################################################################
# gfn-reports module - everything from chapters 2-11 unified
################################################################
locals {
name = "gfn-reports-${var.environment}"
common_tags = merge(var.tags, {
Project = "Compass"
Environment = var.environment
ManagedBy = "terraform"
})
}
# --- ch08: KMS key for at-rest encryption ----------------
resource "aws_kms_key" "reports" {
description = "CMK for ${local.name}"
enable_key_rotation = true
deletion_window_in_days = 30
tags = local.common_tags
}
resource "aws_kms_alias" "reports" {
name = "alias/${local.name}"
target_key_id = aws_kms_key.reports.key_id
}
# --- ch02: IAM execution role ----------------------------
resource "aws_iam_role" "reports" {
name = "${local.name}-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Principal = { Service = "lambda.amazonaws.com" }
Action = "sts:AssumeRole"
}]
})
tags = local.common_tags
}
# --- ch05: S3 raw-telemetry bucket -----------------------
resource "aws_s3_bucket" "raw" {
bucket = "${local.name}-raw"
tags = local.common_tags
}
resource "aws_s3_bucket_versioning" "raw" {
bucket = aws_s3_bucket.raw.id
versioning_configuration { status = "Enabled" }
}
resource "aws_s3_bucket_server_side_encryption_configuration" "raw" {
bucket = aws_s3_bucket.raw.id
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "aws:kms"
kms_master_key_id = aws_kms_key.reports.arn
}
}
}
# --- ch06: DynamoDB aggregated-reports table --------------
resource "aws_dynamodb_table" "reports" {
name = local.name
billing_mode = "PAY_PER_REQUEST"
hash_key = "pk"
attribute { name = "pk"; type = "S" }
server_side_encryption { enabled = true; kms_key_arn = aws_kms_key.reports.arn }
point_in_time_recovery { enabled = var.environment == "prod" }
tags = local.common_tags
}
# --- ch07: SQS intake queue ------------------------------
resource "aws_sqs_queue" "intake" {
name = "${local.name}-intake"
visibility_timeout_seconds = 60
kms_master_key_id = aws_kms_key.reports.id
redrive_policy = jsonencode({
deadLetterTargetArn = aws_sqs_queue.intake_dlq.arn
maxReceiveCount = 5
})
tags = local.common_tags
}
resource "aws_sqs_queue" "intake_dlq" {
name = "${local.name}-intake-dlq"
kms_master_key_id = aws_kms_key.reports.id
tags = local.common_tags
}
# --- ch08: Secrets Manager (DB / 3rd-party creds) ---------
resource "aws_secretsmanager_secret" "app" {
name = "${local.name}/app"
kms_key_id = aws_kms_key.reports.id
tags = local.common_tags
}
# --- ch02 + ch05/6/7/8: bundled inline policy for the role -
resource "aws_iam_role_policy" "reports" {
name = "${local.name}-inline"
role = aws_iam_role.reports.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{ Effect = "Allow", Action = ["s3:GetObject", "s3:PutObject"], Resource = "${aws_s3_bucket.raw.arn}/*" },
{ Effect = "Allow", Action = ["dynamodb:GetItem", "dynamodb:PutItem"], Resource = aws_dynamodb_table.reports.arn },
{ Effect = "Allow", Action = ["sqs:ReceiveMessage", "sqs:DeleteMessage"], Resource = aws_sqs_queue.intake.arn },
{ Effect = "Allow", Action = ["kms:Decrypt", "kms:GenerateDataKey"], Resource = aws_kms_key.reports.arn },
{ Effect = "Allow", Action = ["secretsmanager:GetSecretValue"], Resource = aws_secretsmanager_secret.app.arn },
{ Effect = "Allow", Action = ["logs:CreateLogStream", "logs:PutLogEvents"], Resource = "*" }
]
})
}
# --- ch04: Lambda worker --------------------------------
resource "aws_lambda_function" "worker" {
function_name = "${local.name}-worker"
role = aws_iam_role.reports.arn
handler = "app.handler"
runtime = "python3.12"
filename = var.lambda_zip_path
timeout = 30
memory_size = 512
environment {
variables = {
TABLE_NAME = aws_dynamodb_table.reports.name
QUEUE_URL = aws_sqs_queue.intake.url
SECRET_NAME = aws_secretsmanager_secret.app.name
}
}
tags = local.common_tags
}
# --- ch09: CloudWatch alarm on DLQ depth ----------------
resource "aws_cloudwatch_metric_alarm" "dlq_depth" {
alarm_name = "${local.name}-dlq-depth"
metric_name = "ApproximateNumberOfMessagesVisible"
namespace = "AWS/SQS"
statistic = "Maximum"
period = 60
evaluation_periods = 3
threshold = 5
comparison_operator = "GreaterThanThreshold"
dimensions = { QueueName = aws_sqs_queue.intake_dlq.name }
alarm_actions = var.alarm_sns_topic_arns
tags = local.common_tags
}
# --- ch11: API Gateway HTTP API -------------------------
resource "aws_apigatewayv2_api" "reports" {
name = local.name
protocol_type = "HTTP"
tags = local.common_tags
}
resource "aws_apigatewayv2_integration" "worker" {
api_id = aws_apigatewayv2_api.reports.id
integration_type = "AWS_PROXY"
integration_uri = aws_lambda_function.worker.invoke_arn
payload_format_version = "2.0"
}
# --- ch10: EKS deployment (rendered as a kubernetes_manifest) -
# In the real module this lives behind var.use_eks. For brevity,
# see modules/gfn-reports/eks.tf in the repo.
variable "environment" {
type = string
description = "dev | staging | prod"
}
variable "region" {
type = string
default = "us-east-1"
}
variable "lambda_zip_path" {
type = string
description = "Path to packaged Lambda zip"
}
variable "alarm_sns_topic_arns" {
type = list(string)
default = []
}
variable "tags" {
type = map(string)
default = {}
}
output "role_arn" {
value = aws_iam_role.reports.arn
}
output "raw_bucket_name" {
value = aws_s3_bucket.raw.bucket
}
output "table_name" {
value = aws_dynamodb_table.reports.name
}
output "intake_queue_url" {
value = aws_sqs_queue.intake.url
}
output "api_endpoint" {
value = aws_apigatewayv2_api.reports.api_endpoint
}
output "kms_key_arn" {
value = aws_kms_key.reports.arn
}
terraform {
required_providers {
aws = { source = "hashicorp/aws", version = "~> 5.0" }
}
backend "s3" {
bucket = "compass-tfstate-prod"
key = "envs/prod/gfn-reports.tfstate"
region = "us-east-1"
dynamodb_table = "compass-tfstate-lock"
encrypt = true
}
}
provider "aws" {
region = "us-east-1"
# Assumes prod account via the SSO profile
profile = "compass-prod"
default_tags { tags = { Environment = "prod", Project = "Compass" } }
}
module "reports" {
source = "../../modules/gfn-reports"
environment = "prod"
region = "us-east-1"
lambda_zip_path = "../../build/worker.zip"
alarm_sns_topic_arns = ["arn:aws:sns:us-east-1:333333333333:oncall-pager"]
}
terraform plan output
Terraform will perform the following actions:
# module.reports.aws_kms_key.reports will be created
+ resource "aws_kms_key" "reports" {
+ arn = (known after apply)
+ description = "CMK for gfn-reports-prod"
+ enable_key_rotation = true
+ key_usage = "ENCRYPT_DECRYPT"
}
# module.reports.aws_iam_role.reports will be created
+ resource "aws_iam_role" "reports" {
+ name = "gfn-reports-prod-role"
+ assume_role_policy = jsonencode({ ... })
}
# module.reports.aws_s3_bucket.raw will be created
+ resource "aws_s3_bucket" "raw" {
+ bucket = "gfn-reports-prod-raw"
}
# module.reports.aws_dynamodb_table.reports will be created
# module.reports.aws_sqs_queue.intake will be created
# module.reports.aws_sqs_queue.intake_dlq will be created
# module.reports.aws_secretsmanager_secret.app will be created
# module.reports.aws_iam_role_policy.reports will be created
# module.reports.aws_lambda_function.worker will be created
# module.reports.aws_cloudwatch_metric_alarm.dlq_depth will be created
# module.reports.aws_apigatewayv2_api.reports will be created
# module.reports.aws_apigatewayv2_integration.worker will be created
Plan: 12 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ api_endpoint = (known after apply)
+ intake_queue_url = (known after apply)
+ kms_key_arn = (known after apply)
+ raw_bucket_name = "gfn-reports-prod-raw"
+ role_arn = (known after apply)
+ table_name = "gfn-reports-prod"
12 resources in one apply. The same module + dev tfvars gives you dev; same module + prod tfvars gives you prod. New region or new account? Add an env folder and a backend, point the provider at it, done. That's the payoff for the Terraform work.
In a real multi-account world you would now:
provider.terraform plan as a comment-on-PR; merge to main triggers terraform apply; manual destroy is gated behind an approval workflow.terraform test; wire it into your CI as the canonical deployment path.
If you started this book at chapter 1, you have just travelled a long way. A quick map of where you've been:
| # | Chapter | What you can now do |
|---|---|---|
| 1 | Hello, AWS | Navigate the AWS console, install & configure the CLI, understand accounts/regions/AZs and the SSO + profile model. |
| 2 | IAM & Identity | Read any IAM policy in JSON, design role-based workload identity, debug "AccessDenied", reason about trust policies and cross-account access. |
| 3 | Networking (VPC) | Lay out VPCs, subnets, route tables, NAT/IGW, and security groups vs NACLs - and spot the places Azure VNet habits will mislead you. |
| 4 | Compute | Pick between EC2 / Lambda / ECS / Fargate / EKS based on workload shape, not vibes. |
| 5 | Storage | Use S3 with the right storage class, lifecycle rules, and versioning. Understand EBS / EFS trade-offs vs Azure Files / Managed Disks. |
| 6 | Databases | RDS, Aurora, and DynamoDB - including how partition keys and capacity modes really work and why DynamoDB pricing surprises Cosmos veterans. |
| 7 | Messaging & Events | SQS, SNS, EventBridge, Kinesis - the closest-to-1:1 mapping of any chapter, and where each one earns its place. |
| 8 | Security & Secrets | KMS keys (and their key policies, the AWS-specific gotcha), Secrets Manager rotation, ACM, GuardDuty. |
| 9 | Observability | CloudWatch metrics, logs and alarms; CloudTrail for audit; CloudWatch Logs Insights as the AWS analog to KQL. |
| 10 | Kubernetes on AWS (EKS) | Stand up EKS, wire IRSA (the EKS-IAM bridge), pick between Karpenter and Managed Node Groups, attach AWS Load Balancer Controller. |
| 11 | Serverless patterns | Compose Lambda + API GW + DynamoDB + EventBridge into a coherent whole; recognize when the "serverless tax" stops paying. |
| 12 | IaC & Multi-account | Terraform with proper state locking, when CloudFormation/CDK/Terraform each fit, AWS Organizations + SCPs + Control Tower, and the whole gfn-reports module pulled together. |
The gfn-reports stack is a small but architecturally complete service: a dedicated IAM role running a Lambda that consumes events from an SQS queue, writes raw telemetry to S3, aggregates into DynamoDB, all encrypted with a KMS key, with credentials in Secrets Manager, instrumented with CloudWatch alarms, exposed via an API Gateway, optionally hosted on EKS via IRSA, and packaged as a reproducible Terraform module deployable to dev or prod. If you can build this much from scratch, you can build most of what AWS jobs ask for.
AWS Solutions Architect - Professional (SAP-C02) is the natural next exam. It assumes everything in this book and adds advanced networking (Transit Gateway, PrivateLink topologies), multi-region failover patterns, large-scale cost optimization, and migration scenarios. About 75 questions, 3 hours. Plan for ~120 hours of focused study.
Read Karpenter internals (it's getting better fast), the EKS best-practices guide (eksctl.io/best-practices), and learn the AWS Load Balancer Controller annotations. If your day job is GFN/NVIDIA-flavored, also look at GPU node groups, EFA networking, and Bottlerocket.
Cost Explorer + the AWS Compute Optimizer + Savings Plans + Reserved Instances. The FinOps Foundation framework is the lingua franca; the AWS Well-Architected Tool's Cost Optimization pillar review is free and surprisingly useful.
Route 53 latency-based + health-checks, DynamoDB Global Tables, S3 Cross-Region Replication, Aurora Global Database. Then patterns: active-active, active-passive, pilot-light, warm-standby - and what each actually costs.
Free, self-paced, hands-on. Worth doing: EKS Workshop, Serverless Land patterns, the AWS Workshop catalog generally. They're the closest thing to "lab time on a real account" without burning your own budget.
Lambda Powertools (Python/TypeScript), provisioned concurrency & SnapStart, container-image Lambdas, Lambda extensions, Step Functions for orchestration over plain Lambda chaining. The Serverless Patterns Collection is a goldmine of vetted shapes.