@auth-craft/aws-cf-stack 1.8.0 → 1.10.0

package/.env.example

@@ -155,6 +155,21 @@ LAMBDA_REFRESH_COOKIE_TENANT=refresh_token_tenant
 LAMBDA_REFRESH_COOKIE_SYSTEM=refresh_token_system
 LAMBDA_AUDIT_ENABLED=false
 LAMBDA_DEVICE_TRACKING=true
+
+# ─── Observability cost controls (optional) ──────────────────────────────────
+# CloudWatch Logs are billed for ingestion AND stored volume; retention only affects
+# storage. Short retention in dev is the cheap default.
+#   Unset  → 3 days in dev, 30 days in staging/prod.
+#   Valid  → 1,3,5,7,14,30,60,90,120,150,180,365,400,545,731,1096,1827,2192,2557,
+#            2922,3288,3653  (CloudWatch's fixed set — anything else fails at synth),
+#            or 'infinite' to never expire.
+LAMBDA_LOG_RETENTION_DAYS=
+# X-Ray tracing is billed per recorded trace. Unset → disabled in dev, active otherwise.
+#   active | disabled | pass_through
+LAMBDA_TRACING=
+# Adding missing GSIs to an existing table (one per CloudFormation update): max minutes to
+# wait for each to backfill before giving up (re-running resumes). Unset → 60.
+AUTH_CRAFT_GSI_WAIT_MINUTES=
 LAMBDA_APP_URL=
 LAMBDA_SUPPORT_EMAIL=
 LAMBDA_CORS_ORIGINS=

package/README.md

@@ -87,7 +87,9 @@ all point at the single worker.
 
 - `aws` CLI authenticated (or `AWS_*` creds in env), `cdk` (or `npx cdk`), `wrangler`,
   `node` ≥ 20, `jq`, `curl`.
-- AWS + Cloudflare credentials.
+- AWS + Cloudflare credentials. Besides the CDK deploy roles, the identity running the CLI
+  needs `dynamodb:DescribeTable` on the `<app>-<stage>-<layer>-data` table and
+  `cloudformation:GetTemplate` on the data stack (see [Table schema & GSI rollout](#table-schema--gsi-rollout)).
 
 ## Usage
 
@@ -163,6 +165,75 @@ CF_EDGE_GATE_SECRET="$SNAP_EDGE" CLOUDFLARE_ACCOUNT_ID="$SNAP_CF_ACCT" \
   npx auth-craft-stack all --stage "$STAGE"
 ```
 
+## Table schema & GSI rollout
+
+The table's keys and GSIs come from one source: `TABLE_SCHEMA` in
+`@auth-craft/database-plugin-dynamodb`, snapshotted into `cdk/lib/table-schema.json` at build
+time. The CDK stack, the integration tests and the create-table script all read it, so a deployed
+table can no longer miss an index the repositories query.
+
+CloudFormation creates or deletes **at most one GSI per table update**. A new table gets every GSI
+at once, but an existing table deployed by an older version may lack several. Before the normal
+deploy, `lambda` therefore:
+
+1. `DescribeTable`s the existing table (nothing to do if it doesn't exist yet).
+2. Deploys the data stack with the table's current GSIs **plus the next missing one**.
+3. Waits for that GSI to finish backfilling (`ACTIVE`), then repeats until the schema is complete.
+
+Each GSI takes a few minutes on a populated table; the wait is capped by
+`AUTH_CRAFT_GSI_WAIT_MINUTES` (default 60) and an interrupted run resumes where it stopped.
+
+GSIs are compared by their full definition (keys, key attribute types, projection), not by name.
+The CLI stops before deploying, and prints how to fix it, when:
+
+- the table has a GSI that CloudFormation doesn't know about (created by hand);
+- the stack declares a GSI the table lacks (deleted by hand). The CLI prints the
+  `aws dynamodb update-table` command that recreates it as the stack declares it;
+- the table has a GSI the schema no longer declares;
+- a GSI's definition differs from the schema. CloudFormation would have to replace it, which is
+  two GSI operations in one update. A changed index needs a new name in `TABLE_SCHEMA`;
+- a rollout step finished without adding its GSI.
+
+The GSI subset for each rollout step goes to the CDK app as context (`-c dataGsis=...`), never as
+an environment variable, so nothing in your shell or `--env-file` can shrink the full deploy.
+
+## Observability cost controls (log retention, tracing)
+
+CloudWatch Logs and X-Ray are usually the two line items that make a low-traffic **dev**
+stage cost more than the Lambda itself. Both are configurable, with stage-aware defaults
+that keep dev cheap without touching staging/prod behaviour:
+
+| Setting | Env var / flag | Default (dev) | Default (staging/prod) |
+|---------|----------------|---------------|------------------------|
+| Log retention | `LAMBDA_LOG_RETENTION_DAYS` / `--log-retention-days` | `3` days | `30` days |
+| X-Ray tracing | `LAMBDA_TRACING` / `--tracing` | `disabled` | `active` |
+
+```bash
+# Cheapest dev: keep 1 day of logs, no tracing (tracing is already off by default in dev)
+npx auth-craft-stack lambda --stage dev --log-retention-days 1
+
+# Turn tracing back on in dev while debugging
+npx auth-craft-stack lambda --stage dev --tracing active
+
+# Long retention for an audited prod
+npx auth-craft-stack lambda --stage prod --log-retention-days 365
+```
+
+Notes:
+- Retention accepts only **CloudWatch's fixed set** — `1, 3, 5, 7, 14, 30, 60, 90, 120, 150,
+  180, 365, 400, 545, 731, 1096, 1827, 2192, 2557, 2922, 3288, 3653` — or `infinite` to never
+  expire. Any other value fails at **synth** time with the valid list, so a typo can't
+  silently deploy the wrong retention.
+- Retention caps **storage** cost only; you are still billed for log **ingestion**. To cut
+  ingestion, reduce what the app logs.
+- Changing retention applies to the existing log group too (it's the same CloudFormation
+  resource), so old logs beyond the new window age out — it is not just for new logs.
+- In `prod` the log group has `RemovalPolicy.RETAIN`, so destroying the stack keeps the logs
+  (and the group must be deleted manually before a same-name redeploy). In dev/staging it is
+  destroyed with the stack.
+- `LogRetentionDays` and `TracingMode` are emitted as CDK outputs, so
+  `npx auth-craft-stack outputs` shows what is actually deployed.
+
 ## CORS (per scope + dynamic allowlist)
 
 CORS is resolved **per scope** (system / tenant / customer) — the 3 workers no longer
@@ -326,6 +397,11 @@ time). Bump the version to deploy newer auth-craft.
   already-deployed queue. The stack grants the Lambda `sqs:SendMessage` on that ARN, but it
   can't create a cross-stack queue — deploy your email/queue stack first, then this one.
 
+- **`DATABASE_QUERY_ERROR` from invite flows on a table deployed by ≤ 1.9.0** — the table lacks
+  `GSI_InviteToken` / `GSI_InviteeEmail` / `GSI_InviteePhone`. Deploy `lambda` with ≥ 1.10.0; it
+  adds them one at a time. The Lambda log names the cause:
+  `ValidationException: The table does not have the specified index: …`.
+
 ## Email delivery modes
 
 `LAMBDA_EMAIL_MODE` = `console` (default) | `smtp` | `queue`.
@@ -345,6 +421,9 @@ time). Bump the version to deploy newer auth-craft.
   (ESM, `node:*` external — provided by the Workers runtime), the same way wrangler bundles
   internally. Uses the esbuild binary in the app's `node_modules` (the root `.bin/esbuild`
   symlink can be stale because esbuild is in pnpm `ignoredBuiltDependencies`).
+- **Table schema**: `scripts/sync-table-schema.mjs` writes `cdk/lib/table-schema.json` from
+  `TABLE_SCHEMA` (database-plugin-dynamodb is unpublished, so the CDK app can't import it at the
+  consumer). The JSON is committed; `pnpm test` fails if it's stale (`pnpm sync:table-schema`).
 - **Admin CLI**: esbuilds `packages/auth-admin-cli/src/index.ts` → `assets/admin/cli.mjs`,
   fully inlined (auth-core + plugins + `pg` + aws-sdk). It is **bundled, not a dependency**,
   because `@auth-craft/auth-admin-cli` is `private` and pulls in ~7 workspace packages —
@@ -352,6 +431,5 @@ time). Bump the version to deploy newer auth-craft.
 
 The bundled `cdk/lib/lambda-stack.ts` deploys the Lambda via
 `lambda.Code.fromAsset('../../assets/lambda')` + handler `index.handler` (not `NodejsFunction`),
-so no source build happens at the consumer's deploy time. It is a **copy** of the app's
-`infrastructure/lib/lambda-stack.ts` — keep `cdk/lib/*.ts` in sync until a future refactor
-unifies them.
+so no source build happens at the consumer's deploy time. These `cdk/lib/*.ts` stacks are the
+only CDK definition of the auth deploy — the app's old `infrastructure/` copy was removed.