Chapter 5 of 16 · Human user · Priya

Policies and 403 Troubleshooting

Turn permission denied into a precise capability diagnosis without broadening access blindly.

Chapter contract

Persona: Human user · PriyaPhase: Human Vault user foundationsEstimated time: 40 minutes

Prerequisites:

Chapter 4 complete; know the exact operation and redacted API path being tested.

User-visible outcome:

Priya can diagnose an expected 403 from path and capability evidence instead of requesting broad access.

Learning objectives

Consumer flow

Requested operationstep 1Exact API pathstep 2Policy capabilitystep 3Allow or denystep 4

Ownership boundary

OwnerConsumer-facing responsibility
You ownDefines consumption, validation, reload, and stop-using behavior.
Platform team ownsProvides trusted runtime identity, protected delivery, network, and telemetry.
Vault ownsOperates auth mounts, policies, engines, audit, and recovery.

Core concepts

Capability

An operation such as read, create, update, delete, list, sudo, or patch.

Path rule

A policy stanza matching API paths, not CLI wording.

Default deny

Anything not explicitly allowed is denied.

Audit correlation

Matching a client failure to server audit metadata by time and request context.

Commands and configuration

source learn-vault/.runtime/learner.env
vault token capabilities lockbox-kv/data/orders-api
vault token capabilities lockbox-kv/metadata/orders-api
vault kv metadata get -format=json lockbox-kv/orders-api | jq '.data.current_version'

Secret-safety stop

Stop: Do not solve a 403 by attaching an admin policy. Prove the exact path and capability first.

Failure drill

Symptom403 permission denied on a KV read.
Likely causesPolicy covers the friendly path instead of KV v2 data, role mapping is wrong, or list targets the wrong route.
Inspect safelyQuery capabilities for the exact API path and capture status plus timestamp.
Do not printToken, request payload, secret response, or raw audit record.

Break it / fix it

Break it: Attempt the intentionally denied KV delete.

Show diagnosis and fix

Fix it: Query capabilities for the exact API path; retain the narrow read policy.

Escalate with time, endpoint, auth path, role, mount, namespace when relevant, status code, request ID, and sanitized error class. Never attach a credential or response body.

Try it

Bootstrap boundary

labs/bootstrap owns the disposable server, auth mounts, policies, roles, and engines; learner actions begin after setup.

Cost:

$0; local containers only

Starting state:

Run labs/bootstrap/setup.sh and source .runtime/learner.env unless the chapter lab says AppRole alone is sufficient.

User actions:

Prove an allowed metadata operation and a denied delete, then inspect the bootstrap policy source.

source learn-vault/.runtime/learner.env
test "$(vault token capabilities lockbox-kv/metadata/orders-api)" = read
if vault kv delete lockbox-kv/orders-api >/dev/null 2>&1; then
  echo "unexpected delete permission"; exit 1
else
  echo "delete denied as expected"
fi
sed -n '/lockbox-kv\/metadata/,/}/p' learn-vault/labs/bootstrap/policies/lockbox-learner.hcl

Success criteria:

Complete every item in the Verify section using metadata-only evidence; no secret value appears in terminal output or tracked files.

Troubleshooting:

Use the Failure drill and Break it / fix it evidence fields. Stop before broadening policy, weakening identity or TLS checks, or copying secret-bearing diagnostics.

Verify

  1. Metadata reports read capability.
  2. Delete is denied and the KV value remains.
  3. The policy uses the KV v2 metadata route.
Evidence rule: retain only status, policy names, TTLs, versions, timestamps, fingerprints, and error classes. Never retain secret values.

Cleanup

No Vault data changed. Unset VAULT_TOKEN if the shell will be shared.

Project Lockbox increment

The Lockbox access request names exact API paths and capabilities, making review and diagnosis reproducible.

Quick check

What happens when no policy matches?

Deny.

Why can lockbox-kv/orders-api fail?

KV v2 operations include data or metadata path segments.

Where is list granted for KV v2?

On the metadata path.

What belongs in a change request?

Identity role, exact redacted API path, capability, and reason.

Gotchas

Recap and next

A good 403 investigation narrows the contract; it does not widen policy until the error disappears.

Primary references

Continue →