serverless-ircd 0.1.0 → 0.3.0

package/docs/AWS-Deployment.md

@@ -0,0 +1,1186 @@
+# AWS Deployment Guide
+
+End-to-end guide for deploying ServerlessIRCd to AWS (Phase 4 of the
+project plan). Covers prerequisites, first-time AWS setup, local
+development, staging and production deploys, configuration reference,
+DynamoDB capacity planning, region strategy, cost notes, CI/CD, and
+troubleshooting.
+
+Cross-reference: the AWS Adapter Architecture doc
+(`../../ServerlessIRCd/docs/AWS-Adapter-Architecture.md`) for the
+diagrams referenced throughout, and the Cloudflare Deployment Guide
+(`../../ServerlessIRCd/docs/Cloudflare-Deployment-Guide.md`) for the
+equivalent flow on the other supported platform. The shared
+`irc-core`/`irc-server` packages and the contract test suite are
+identical between platforms; only the I/O shell (this stack) differs.
+
+Key in-repo paths cited below:
+
+| Path                                              | What                                          |
+|---------------------------------------------------|-----------------------------------------------|
+| `apps/aws-stack/src/aws-stack.ts`                 | CDK stack — tables, Lambdas, APIGW, schedules.|
+| `apps/aws-stack/bin/aws.ts`                       | CDK app entry. Stack id is `IrcAwsStack`.     |
+| `apps/aws-stack/cdk.json`                         | `app = tsx bin/aws.ts`.                        |
+| `apps/aws-stack/scripts/smoke.mjs`                | CI + local smoke e2e.                         |
+| `apps/aws-stack/tests/stack.test.ts`              | Synth-time assertions (cross-check for facts).|
+| `packages/aws-adapter/src/cdk-table-defs.ts`      | Five DynamoDB table shapes.                   |
+| `packages/aws-adapter/src/config-loader.ts`       | Lambda env → `ServerConfig` mapping.          |
+| `packages/aws-adapter/src/handlers/index.ts`      | Lambda entry (`handler`, `sweeperHandler`, `pingCheckerHandler`). |
+| `.github/workflows/deploy-aws.yml`                | Staging deploy + smoke e2e CI.                |
+
+**Acceptance criterion:** a new contributor can deploy their own
+staging instance following only this doc.
+
+---
+
+## 1. What gets deployed
+
+A single CDK v2 stack (`IrcAwsStack`) provisions the entire AWS
+substrate. There are no static assets — CloudFront and S3 are not part
+of this deployment. Every resource is regional.
+
+```
+   ┌──────────────────────────────────────────────────────────────────┐
+   │  CDK stack: IrcAwsStack  (apps/aws-stack/src/aws-stack.ts)       │
+   │                                                                  │
+   │  ┌─ WebSocketApi (AWS::ApiGatewayV2::Api)  ───────────────────┐  │
+   │  │   routes: $connect  $disconnect  $default                  │  │
+   │  │   → all three wired to IrcHandler                          │  │
+   │  │   stage: prod (auto-deploy)                                │  │
+   │  └────────────────────────────────────────────────────────────┘  │
+   │                           │                                      │
+   │                           ▼  WebSocket event per frame           │
+   │  ┌─ Lambda: IrcHandler ───────────────────────────────────────┐  │
+   │  │   handler export `handler` (Node 20, esbuild bundle)       │  │
+   │  │   entry: packages/aws-adapter/src/handlers/index.ts        │  │
+   │  │   dispatches on event.requestContext.routeKey              │  │
+   │  └────────────────────────────────────────────────────────────┘  │
+   │                                                                  │
+   │  ┌─ Lambda: IrcSweeper ───────────────────────────────────────┐  │
+   │  │   handler export `sweeperHandler`                          │  │
+   │  │   EventBridge schedule: rate(5 minutes)                    │  │
+   │  │   Scans Connections for rows past hard TTL; cleans up.     │  │
+   │  └────────────────────────────────────────────────────────────┘  │
+   │                                                                  │
+   │  ┌─ Lambda: IrcPingChecker ───────────────────────────────────┐  │
+   │  │   handler export `pingCheckerHandler`                      │  │
+   │  │   EventBridge schedule: rate(1 minute)                     │  │
+   │  │   Sends PING to idle connections; disconnects no-PONG.     │  │
+   │  └────────────────────────────────────────────────────────────┘  │
+   │                                                                  │
+   │  ┌─ DynamoDB (on-demand) ─────────────────────────────────────┐  │
+   │  │   Connections    (PK connectionId,     TTL idleSince)      │  │
+   │  │   ChannelMeta    (PK channelName)                          │  │
+   │  │   ChannelMembers (PK channelName, SK connectionId)         │  │
+   │  │   Nicks          (PK nickLower)                            │  │
+   │  │   Accounts       (PK account)                              │  │
+   │  └────────────────────────────────────────────────────────────┘  │
+   │                                                                  │
+   │  CfnOutput: ConnectUrl     = stage.url         (wss://…)         │
+   │  CfnOutput: ManagementUrl  = stage.callbackUrl (https://…)       │
+   └──────────────────────────────────────────────────────────────────┘
+```
+
+All IRC protocol logic lives in the shared `irc-core` reducers and the
+`irc-server` `ConnectionActor`. The Lambdas are a thin I/O shell: they
+forward WebSocket events into the actor and interpret the resulting
+`Effect[]` against DynamoDB and the API Gateway Management API. No IRC
+state is held in the Lambda process between invocations.
+
+The three Lambda functions share a single bundle (esbuild tree-shakes
+per handler export at synth time). They run on `nodejs20.x` and exclude
+`@aws-sdk/*` from the bundle because the AWS SDK v3 ships with the
+Lambda Node 20 runtime.
+
+Stack outputs (`CfnOutput`):
+
+| Output          | Value                       | Purpose                                            |
+|-----------------|-----------------------------|----------------------------------------------------|
+| `ConnectUrl`    | `wss://<id>.execute-api.<region>.amazonaws.com/prod` | WebSocket endpoint IRC clients dial. |
+| `ManagementUrl` | `https://<id>.execute-api.<region>.amazonaws.com/prod` | HTTPS endpoint for `ApiGatewayManagementApi.postToConnection`. |
+
+The same CloudFormation template serves both staging and production.
+The staging/production distinction today is which AWS account and
+region the stack is deployed into — there is no per-environment stack
+parameter (see §6 and §7.4 for the implications).
+
+---
+
+## 2. Prerequisites
+
+| Requirement         | Version / detail                                              |
+|---------------------|---------------------------------------------------------------|
+| Node.js             | ≥ 20 (matches CI; `engines.node` in root `package.json` and in `apps/aws-stack/package.json`). |
+| pnpm                | 9.x (`packageManager: pnpm@9.15.9` in root `package.json`).   |
+| AWS account         | IAM-permissioned to create CloudFormation stacks, IAM roles, API Gateway v2, Lambda, DynamoDB tables, and EventBridge rules. |
+| AWS credentials     | `aws configure`, `AWS_PROFILE`, or SSO. CDK reads the standard SDK chain. |
+| CDK v2              | Comes from `apps/aws-stack/devDependencies` (`aws-cdk ^2.160.0`) — no global install needed. Use `pnpm cdk` (or `pnpm cdk:synth`) so the local binary wins over any globally-installed CDK. |
+| CDK bootstrap       | `cdk bootstrap` must have been run **once** in the target account/region (see §3.3). |
+| Git checkout        | Clean working tree on `main` for production deploys.          |
+
+Confirm the local environment:
+
+```bash
+node --version          # v20.x or newer
+pnpm --version          # 9.x
+aws --version           # aws-cli/2.x (any 2.x is fine)
+pnpm install            # from repo root, installs the whole workspace
+pnpm build              # builds irc-core, irc-server, aws-adapter, aws-stack
+```
+
+`pnpm build` must succeed before `cdk deploy`: the `NodejsFunction`
+construct transpiles the handler entry with esbuild at synth time, and
+esbuild resolves `@serverless-ircd/*` workspace packages via their
+compiled `dist/` outputs.
+
+---
+
+## 3. First-time AWS setup
+
+These steps are done once per AWS account.
+
+### 3.1 Pick a region
+
+API Gateway WebSocket endpoints, DynamoDB, EventBridge Scheduler, and
+Lambda are all regional. WebSocket clients pay round-trip time to the
+region, so pick the one closest to your user base (see §10 for the
+full region discussion). Set it as your default:
+
+```bash
+export AWS_REGION=us-east-1   # or your nearest region
+```
+
+CDK picks the region up via `CDK_DEFAULT_REGION` (set automatically by
+the CDK CLI from the SDK chain).
+
+### 3.2 Configure credentials
+
+Three equivalent options:
+
+```bash
+# Static access key (simplest):
+aws configure
+# → prompts for Access Key ID, Secret Key, region, output format.
+
+# Named profile:
+export AWS_PROFILE=my-profile
+
+# SSO:
+aws sso login --profile my-sso-profile
+export AWS_PROFILE=my-sso-profile
+```
+
+Verify:
+
+```bash
+aws sts get-caller-identity
+```
+
+### 3.3 Bootstrap CDK
+
+CDK needs an S3 bucket + a few IAM roles per account/region to hold
+synthesized CloudFormation assets. Bootstrap once:
+
+```bash
+cd apps/aws-stack
+pnpm cdk bootstrap
+# → creates a CFN stack named CDKToolkit with the bootstrap bucket.
+```
+
+If you skip this, the first `cdk deploy` fails with a bucket-not-found
+error (see §13.1). Bootstrap is idempotent — re-running on an
+already-bootstrapped account is a no-op.
+
+### 3.4 Optional: set up OIDC for CI
+
+If you intend to use the GitHub Actions deploy workflow (§12) without
+long-lived access keys, configure GitHub OIDC:
+
+1. Create an IAM identity provider for `token.actions.githubusercontent.com`.
+2. Create a role with a trust policy granting
+   `sts:AssumeRoleWithWebIdentity` to your repo's `ref:refs/heads/main`.
+3. Attach a permission policy broad enough to manage the stack (IAM,
+   API Gateway v2, Lambda, DynamoDB, EventBridge, CloudFormation).
+4. Note the role ARN — it becomes the `AWS_DEPLOY_ROLE_ARN` secret in
+   CI. Static keys (`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`) are
+   the fallback; the workflow supports both.
+
+---
+
+## 4. Local development
+
+Unlike the Cloudflare adapter (which has `wrangler dev` against the
+real `workerd` runtime), the AWS adapter does not yet ship a
+hot-reload local Lambda/API Gateway emulator. The local dev loop is:
+
+1. **Synth-time assertions** (default `pnpm test`) — fast, no Docker.
+2. **Unit tests** for the handler, runtime, and access patterns under
+   `packages/aws-adapter/tests/`.
+3. **Optional localstack validation** — gated, requires Docker.
+
+### 4.1 Synth-time tests
+
+From the repo root:
+
+```bash
+pnpm --filter @serverless-ircd/aws-stack test
+# or, equivalently:
+pnpm test     # runs the whole workspace including aws-stack
+```
+
+These execute `Template.fromStack(...)` in-process — no AWS
+credentials, no Docker. Every assertion in
+`apps/aws-stack/tests/stack.test.ts` corresponds to a fact this guide
+states: five on-demand tables, three Node 20 Lambdas, the
+`$connect`/`$disconnect`/`$default` routes, the `prod` stage, the two
+`rate(...)` schedules, the two CfnOutputs, and least-privilege IAM
+(no `Action: "*"` or `Resource: "*"`).
+
+### 4.2 Synthesize the CloudFormation template
+
+```bash
+pnpm --filter @serverless-ircd/aws-stack cdk:synth
+# → writes cdk.out/<stack>.template.json
+```
+
+Inspecting `cdk.out/` is the fastest way to see exactly what will be
+deployed without touching AWS.
+
+### 4.3 Optional: validate against localstack
+
+A gated test (`apps/aws-stack/tests/localstack.test.ts`) runs
+`cdklocal synth` against a real localstack container and re-asserts
+the five-table shape. It is **skipped unless `LOCALSTACK=1` is set** so
+CI never depends on Docker. To run it locally:
+
+```bash
+# 1. Start localstack (requires Docker):
+docker run --rm -d -p 4566:4566 localstack/localstack
+
+# 2. Install the localstack-aware CDK CLI globally:
+npm install -g aws-cdk-local
+
+# 3. Point the AWS SDK at localstack (cdklocal reads these):
+export AWS_ACCESS_KEY_ID=test
+export AWS_SECRET_ACCESS_KEY=test
+export AWS_DEFAULT_REGION=us-east-1
+
+# 4. Run the gated test:
+LOCALSTACK=1 pnpm --filter @serverless-ircd/aws-stack test
+```
+
+### 4.4 Running the smoke script locally
+
+The smoke e2e (`apps/aws-stack/scripts/smoke.mjs`) defaults to
+`ws://localhost:8787`, inherited from the Cloudflare adapter's dev
+port. To smoke-test the AWS adapter locally you would need to stand up
+an API Gateway WebSocket emulator in front of the Lambda handler
+(sam-cli + localstack can do this); that wiring is not preconfigured.
+In practice, local iteration uses the unit + synth tests, and the
+smoke script is run against a deployed staging URL (§5.2) or against
+the Cloudflare adapter's local dev server when you're iterating on the
+shared protocol layer.
+
+---
+
+## 5. Deploy staging
+
+Staging is the environment CI deploys on every push to `main`; you can
+also deploy it manually from a clean checkout.
+
+### 5.1 Deploy from your machine
+
+```bash
+# From the repo root:
+pnpm deploy:aws:staging
+# Equivalent to:
+#   pnpm --filter @serverless-ircd/aws-stack deploy:staging
+#   → cdk deploy --all --require-approval never
+```
+
+The first deploy of a given account/region will:
+
+1. Upload the esbuild-bundled handler asset to the CDK bootstrap S3
+   bucket.
+2. Create the `IrcAwsStack` CloudFormation stack.
+3. Provision five DynamoDB tables, three Lambda functions, one API
+   Gateway WebSocket API, one `prod` stage, two EventBridge rules,
+   least-privilege IAM roles, and the two stack outputs.
+4. Print the deployed URL on success — look for the `ConnectUrl`
+   output, e.g.
+   `wss://abc123.execute-api.us-east-1.amazonaws.com/prod`.
+
+A typical first deploy takes 2–4 minutes (DynamoDB table creation and
+API Gateway stage auto-deploy dominate).
+
+### 5.2 Verify with the smoke script
+
+```bash
+node apps/aws-stack/scripts/smoke.mjs \
+  --url wss://abc123.execute-api.us-east-1.amazonaws.com/prod
+```
+
+The script replays CONNECT → NICK/USER → JOIN #smoke → PRIVMSG → QUIT
+and asserts the server emits `001`, `376`, `353`, `366`, and closes
+the socket. Expected output on success:
+
+```json
+{"ts":"...","level":"info","msg":"aws-stack smoke e2e passed","url":"wss://..."}
+```
+
+On failure the script writes a `level: "error"` JSON line and exits
+non-zero, which is what the CI workflow gates on.
+
+### 5.3 Connect a real IRC client
+
+Point any RFC-compliant WebSocket-aware IRC client at the deployed
+`wss://` URL. For WeeChat:
+
+```
+/server add ircd abc123.execute-api.us-east-1.amazonaws.com/443
+/set irc.server.ircd.ssl on
+/connect ircd
+/join #test
+```
+
+For a quick text-only check, the `apps/local-cli` and
+`tools/tcp-ws-forwarder` packages in the parent repo bridge a plain
+TCP IRC client to a WebSocket endpoint.
+
+---
+
+## 6. Deploy production
+
+The repo-level shortcut is:
+
+```bash
+pnpm deploy:aws:prod
+# Equivalent to:
+#   pnpm --filter @serverless-ircd/aws-stack deploy:prod
+#   → cdk deploy --all --require-approval never
+```
+
+**There is no separate production stack.** `deploy:prod` and
+`deploy:staging` run the identical CDK command — both deploy a stack
+named `IrcAwsStack` (hardcoded in `apps/aws-stack/bin/aws.ts`). The
+staging/production distinction is **which AWS account and region the
+stack is deployed into**, controlled by the credentials and
+`AWS_REGION` in the environment.
+
+For a real production deployment, before your first `pnpm deploy:aws:prod`:
+
+1. **Use a dedicated AWS account** (or at least a dedicated region) for
+   production. Do not point production credentials at the same stack
+   CI is deploying to on every push.
+2. **Change `RemovalPolicy.DESTROY` to `RemovalPolicy.RETAIN`** on the
+   five DynamoDB tables (see §9.4). The committed stack deletes all
+   data on `cdk destroy` — intentional for staging, dangerous for
+   production.
+3. **Override the server identity** (`SERVER_NAME`, `NETWORK_NAME`,
+   `MOTD`) — see §7.1. The committed defaults are placeholders.
+4. **Pick a region close to your users** (see §10).
+
+There is **no CI auto-deploy to production**. The deploy workflow
+(`deploy-aws.yml`) only triggers on push to `main` and targets
+staging. Production deploys are always manual from a clean checkout.
+
+---
+
+## 7. Configuration reference
+
+Server identity (`SERVER_NAME`, `NETWORK_NAME`, `MOTD`) and the logical
+environment name are CDK construct props on `IrcAwsStack`
+(`apps/aws-stack/src/aws-stack.ts`); pass them when instantiating the
+stack. All other deployment knobs still live in the stack source.
+Unlike the Cloudflare adapter (where config lives in a declarative
+`wrangler.toml`), AWS config is code — non-identity overrides require
+editing the stack and re-deploying.
+
+### 7.1 Server identity (construct props)
+
+The three identity values are sourced from `IrcStackProps` and joined
+to the env vars the Lambda config loader reads. `motdLines` is a
+`string[]`; the stack joins it with `\n` so the loader splits it back
+into one entry per IRC numeric (`375` / `372` / `376`):
+
+```ts
+// apps/aws-stack/src/aws-stack.ts (excerpt)
+const motd = (props.motdLines ?? DEFAULT_MOTD_LINES).join('\n');
+handler.addEnvironment('SERVER_NAME', serverName);
+handler.addEnvironment('NETWORK_NAME', networkName);
+handler.addEnvironment('MOTD', motd);
+```
+
+These three values are injected into the `IrcHandler` and
+`IrcPingChecker` Lambda environments at synth time. To override them,
+pass the construct props when creating the stack:
+
+```ts
+new IrcAwsStack(app, 'IrcAwsStack-production', {
+  environmentName: 'production',
+  serverName: 'irc.my.net',
+  networkName: 'MyNet',
+  motdLines: ['Welcome to my network.', 'Be excellent to each other.'],
+});
+```
+
+| Env var         | Purpose                                       | Committed default                              |
+|-----------------|-----------------------------------------------|------------------------------------------------|
+| `SERVER_NAME`   | Server name sent in `001`/`005` numerics.     | `irc.example.com` (placeholder — override!)    |
+| `NETWORK_NAME`  | Network label in `005 NETWORK=…`.             | `ExampleNet`                                   |
+| `MOTD`          | Message-of-the-day. **Delimiter: `\n`** — the loader splits the value on every newline into one MOTD line. | Two-line banner (see note below). |
+
+> **MOTD delimiter contract:** the config loader
+> (`packages/aws-adapter/src/config-loader.ts`) splits `MOTD` on `\n`
+> to form the multi-line MOTD array. The CDK stack's `motdLines`
+> construct prop is a `string[]` joined with `\n`, so each array entry
+> becomes its own line and round-trips back through the loader into the
+> expected number of `motdLines`. Do **not** use a comma (or any other
+> character) as a separator — the loader treats it as literal line
+> content, collapsing the MOTD into a single line.
+
+### 7.2 Lambda runtime config (injected by CDK, do not edit)
+
+These env vars are wired automatically by the stack at synth time. Do
+not hand-set them in the console — the next `cdk deploy` will
+overwrite them.
+
+| Env var                | Source                                    | Consumed by                          |
+|------------------------|-------------------------------------------|--------------------------------------|
+| `MANAGEMENT_URL`       | `stage.callbackUrl` (CfnOutput `ManagementUrl`) | `IrcHandler`, `IrcPingChecker` (builds `ApiGatewayManagementApi` for `postToConnection`). |
+| `CONNECTIONS_TABLE`    | `Connections` (physical table name)       | All three Lambdas (`tablesConfigFromEnv`). |
+| `CHANNELMETA_TABLE`    | `ChannelMeta`                             | All three Lambdas.                   |
+| `CHANNELMEMBERS_TABLE` | `ChannelMembers`                          | All three Lambdas.                   |
+| `NICKS_TABLE`          | `Nicks`                                   | All three Lambdas.                   |
+| `ACCOUNTS_TABLE`       | `Accounts`                                | All three Lambdas.                   |
+| `AWS_REGION`           | Injected by the Lambda runtime, not CDK.  | `buildDepsFromEnv` (constructs the DynamoDB client). |
+
+The `tablesConfigFromEnv` helper
+(`packages/aws-adapter/src/handlers/index.ts:226`) throws on cold
+start if any of the five `<NAME>_TABLE` vars is missing. The smoke
+timeout in §13.2 traces back to this.
+
+### 7.3 Optional config-loader env vars (read if present)
+
+The shared config loader (`packages/aws-adapter/src/config-loader.ts`)
+also reads these optional env vars and feeds them through the shared
+Zod `ServerConfig` schema. **The committed CDK stack does not inject
+any of them** — to enable one, add an `addEnvironment(...)` line in
+`aws-stack.ts` and re-deploy.
+
+| Env var                   | Type    | Schema field          |
+|---------------------------|---------|-----------------------|
+| `MAX_CLIENTS`             | int     | `maxClients`          |
+| `CHANNEL_PREFIXES`        | string  | `channelPrefixes`     |
+| `OPER_USER`               | string  | `operCreds[0].user`   |
+| `OPER_PASSWORD`           | string  | `operCreds[0].password` |
+| `MAX_CHANNELS_PER_USER`   | int     | `maxChannelsPerUser`  |
+| `MAX_TARGETS_PER_COMMAND` | int     | `maxTargetsPerCommand`|
+| `NICK_LEN`                | int     | `nickLen`             |
+| `CHANNEL_LEN`             | int     | `channelLen`          |
+| `TOPIC_LEN`               | int     | `topicLen`            |
+| `MAX_LIST_ENTRIES`        | int     | `maxListEntries`      |
+| `QUIT_MESSAGE`            | string  | `quitMessage`         |
+| `SASL_ACCOUNTS`           | string  | `saslAccounts`        |
+
+`SASL_ACCOUNTS` is the legacy/config fallback (newline-delimited
+`username:password` pairs); it is used only when the `Accounts` table is
+empty. When the table has rows it is authoritative — see §8.1.
+
+Invalid values cause a readable boot-time error — the Lambda will
+throw on the first invocation (cold start). Test changes locally with
+`pnpm --filter @serverless-ircd/aws-adapter test`.
+
+### 7.4 No per-environment stack parameter
+
+There is no `stageName` or `environmentType` parameter on the CDK
+stack. The stack name (`IrcAwsStack`), table names
+(`Connections`, `ChannelMeta`, `ChannelMembers`, `Nicks`, `Accounts`),
+API Gateway stage name (`prod`), and Lambda construct ids are all
+hardcoded. To run staging and production in **the same AWS account**,
+you would need to fork the stack to parameterize these names; the
+recommended path is to use separate accounts (or separate regions) for
+staging vs production. This is the same architectural choice the
+Cloudflare adapter makes (staging and production are separate Worker
+namespaces).
+
+### 7.5 Lambda runtime and bundling
+
+| Knob              | Value                                                      |
+|-------------------|------------------------------------------------------------|
+| Runtime           | `nodejs20.x` (`Runtime.NODEJS_20_X`).                      |
+| Entry             | `packages/aws-adapter/src/handlers/index.ts` (computed by `handlerEntry()` in `aws-stack.ts`). |
+| Handler exports   | `IrcHandler` → `handler`; `IrcSweeper` → `sweeperHandler`; `IrcPingChecker` → `pingCheckerHandler`. |
+| Bundler           | esbuild (via `NodejsFunction`).                            |
+| External modules  | `@aws-sdk/*` (ships with the runtime; keeps the bundle small). |
+| Memory / timeout  | CDK defaults (128 MB / 3 s). Override via `memorySize`/`timeout` props if you observe cold-start or fanout timeouts. |
+
+### 7.6 API Gateway stage
+
+| Knob              | Value                                                      |
+|-------------------|------------------------------------------------------------|
+| Stage name        | `prod` (hardcoded as `STAGE_NAME` in `aws-stack.ts`).     |
+| Auto-deploy       | `true` (changes propagate without a manual deployment).   |
+| Auth on `$connect`| None today (no authorizer). See §13.5.                    |
+| Route responses   | Default (`$connect` returns 200 → upgrade; `$default` and `$disconnect` are fire-and-forget). |
+
+---
+
+## 8. Secrets
+
+**As of v1 the Lambda consumes no Secrets-Manager/SSM bindings.** Server
+password and oper creds are handled by later hardening work; when they
+ship, the natural homes are AWS Secrets Manager or SSM Parameter Store
+(the AWS adapter architecture doc calls both out as supporting
+services). SASL account credentials, however, **are** persisted — in the
+`Accounts` DynamoDB table (see §8.1).
+
+When secrets land, the typical pattern will be:
+
+1. Create the secret in Secrets Manager (or a `SecureString`
+   parameter in SSM).
+2. Grant the Lambda role `secretsmanager:GetSecretValue` (or
+   `ssm:GetParameter`) scoped to that one secret ARN.
+3. Read it inside `buildDepsFromEnv` (cold start) — never inside the
+   per-frame hot path.
+
+```bash
+aws secretsmanager create-secret \
+  --name serverless-ircd/staging/SERVER_PASSWORD \
+  --secret-string "$(openssl rand -base64 32)"
+```
+
+### 8.1 SASL accounts (`Accounts` table + seed tooling)
+
+SASL PLAIN credentials live in the `Accounts` DynamoDB table as
+`HashedAccountCredential` rows — `{ account, algorithm, salt, hash }`,
+all strings, with scrypt hashes encoded as base64 (never plaintext).
+At Lambda cold start, `buildDepsFromEnv` → `resolveAccountStore`
+scans the table once and pre-loads every row into a
+`DynamoAccountStore`; the synchronous `verify()` called from the SASL
+reducer works against that in-memory snapshot. The snapshot is rebuilt
+on every cold start, so account changes take effect within one cold
+start (seconds to minutes).
+
+**Precedence** (`resolveAccountStore`, in `account-store.ts`):
+
+1. The `Accounts` table is **authoritative when it has ≥1 row** — the
+   table wins and the config seed is ignored.
+2. When the table is empty, the adapter falls back to the
+   `SASL_ACCOUNTS` env var (newline-delimited `username:password` pairs,
+   see §7.3) so existing deployments keep working without migrating.
+3. When neither source has accounts, no `AccountStore` is bound and
+   `AUTHENTICATE PLAIN` returns `904 ERR_SASLFAIL`.
+
+**Seed / CRUD tooling.** Provision accounts by writing
+`HashedAccountCredential` rows produced by `hashAccountCredential`
+(scrypt, random salt, base64-encoded). The adapter exports
+`putAccountCredential(docClient, tableName, username, password)`
+for this — a repeat call for the same `username` overwrites the row
+(the PK is `account`). A minimal seeding snippet (Node 20, from the repo
+root):
+
+```bash
+node --input-type=module -e '
+  import { createDynamoDocumentClient, putAccountCredential } from "@serverless-ircd/aws-adapter";
+  const dynamo = createDynamoDocumentClient({ region: "us-east-1" });
+  await putAccountCredential(dynamo, "StagingAccounts", "alice", "change-me");
+  console.log("seeded alice");
+'
+```
+
+A repo-local seed script (`tools/seed-aws-accounts.ts`) wraps the same
+calls for batch provisioning:
+
+```bash
+# Seed against DynamoDB Local:
+npx tsx tools/seed-aws-accounts.ts \
+  --table StagingAccounts \
+  --endpoint http://localhost:8000 \
+  --accounts alice:s3cret bob:password2
+
+# Or from a file (newline-delimited username:password):
+npx tsx tools/seed-aws-accounts.ts \
+  --table StagingAccounts \
+  --region us-east-1 \
+  --file accounts.txt
+```
+
+For a one-off `aws-cli` write, hash the password out-of-band and put the
+row directly:
+
+```bash
+# Hash with the adapter, then write the row yourself:
+aws dynamodb put-item \
+  --table-name StagingAccounts \
+  --item '{"account":{"S":"alice"},"algorithm":{"S":"scrypt"},"salt":{"S":"<base64-salt>"},"hash":{"S":"<base64-hash>"}}'
+```
+
+> The `Accounts` table is granted to all three Lambdas (`handler`,
+> `sweeper`, `pingChecker`) via `grantReadWriteData`; only the handler
+> reads it at cold start. The sweeper/ping-checker never touch SASL
+> state, but sharing the grant keeps the IAM story uniform.
+
+**Never** put credentials, password hashes, or API tokens in
+`aws-stack.ts`, in Lambda env vars, or in committed files. Lambda env
+vars are visible in the console and in CloudFormation describe output;
+they are not secret.
+
+The CI deploy uses **GitHub Actions secrets** plus an optional repo
+variable (see §12):
+
+| Secret                  | Used by           | Purpose                                                       |
+|-------------------------|-------------------|---------------------------------------------------------------|
+| `AWS_ACCESS_KEY_ID`     | `deploy-aws.yml`  | Static access key for CDK deploy. Required unless OIDC is used. |
+| `AWS_SECRET_ACCESS_KEY` | `deploy-aws.yml`  | Companion secret key. Required unless OIDC is used.           |
+| `AWS_DEPLOY_ROLE_ARN`   | `deploy-aws.yml`  | Optional. When set, the job assumes this role via OIDC web identity instead of using static keys. |
+| `AWS_REGION`            | `deploy-aws.yml`  | Region to deploy into (e.g. `us-east-1`).                     |
+
+| Variable        | Purpose                                                              |
+|-----------------|---------------------------------------------------------------------|
+| `AWS_SMOKE_URL` | Optional. `wss://` URL for the smoke e2e step. If unset, the workflow falls back to reading the stack's `ConnectUrl` CloudFormation output via the AWS CLI; if that also fails the smoke step is skipped with a warning. |
+
+Configure all of these under repo **Settings → Secrets and variables
+→ Actions**.
+
+---
+
+## 9. DynamoDB capacity planning
+
+### 9.1 All five tables run on-demand
+
+Every table is created with `BillingMode.PAY_PER_REQUEST` (asserted in
+`apps/aws-stack/tests/stack.test.ts:45`). On-demand pricing charges
+per request:
+
+- **Write request units (WRU)** — one WRU = one write of up to 1 KB
+  (or 2 WRU for a strongly-consistent write of up to 1 KB). A
+  conditional write that fails (e.g. `attribute_not_exists` rejects a
+  nick collision) still costs 1 WRU.
+- **Read request units (RRU)** — one RRU = one eventually-consistent
+  read of up to 4 KB (or 2 RRU for strongly consistent).
+
+There is no minimum spend and no capacity to plan at deploy time.
+This is intentional for v1 — it removes a class of
+`ProvisionedThroughputExceededException` failure modes. See §11 for
+the cost trade-off and when to consider provisioned capacity.
+
+### 9.2 Schema and access patterns
+
+Five tables (defined in
+`packages/aws-adapter/src/cdk-table-defs.ts`):
+
+| Table           | PK (HASH)         | SK (RANGE)       | TTL         | Hottest access pattern                                   |
+|-----------------|-------------------|------------------|-------------|----------------------------------------------------------|
+| `Connections`   | `connectionId`    | —                | `idleSince` | `GetItem`/`UpdateItem` on every frame; `Scan` by sweeper. |
+| `ChannelMeta`   | `channelName`     | —                | —           | `GetItem`/`UpdateItem` on MODE/TOPIC/KICK. Low volume.   |
+| `ChannelMembers`| `channelName`     | `connectionId`   | —           | **`Query` on every channel PRIVMSG for fanout.** Hottest. |
+| `Nicks`         | `nickLower`       | —                | —           | `PutItem` (conditional) on NICK; `GetItem` on PRIVMSG to a nick. |
+| `Accounts`      | `account`         | —                | —           | `GetItem` on SASL AUTHENTICATE. Cold most of the time.   |
+
+Uniqueness invariants are enforced structurally (not in application
+code):
+
+- `Nicks.PK = nickLower` — conditional `PutItem` with
+  `attribute_not_exists(PK)` makes nick reservation atomic across
+  concurrent Lambdas. Two clients racing on `Alice` both get evaluated
+  against the same row; exactly one wins.
+- `ChannelMembers` PK+SK composite — `TransactWriteItems` for JOIN/PART
+  updates `Connections.joined` and `ChannelMembers` in one atomic
+  write (up to 100 items per transaction, the AWS limit).
+- `Accounts` row schema — `{ account (PK), algorithm, salt, hash }`,
+  all strings. `algorithm` is `'scrypt'`; `salt` and `hash` are
+  base64-encoded. Never plaintext. SASL PLAIN verifies against this
+  hash server-side. Seed rows with `putAccountCredential` (see §8.1).
+
+### 9.3 The hottest pattern: channel fanout
+
+Channel PRIVMSG is the dominant cost driver. There is no long-lived
+per-channel process (contrast with the Cloudflare adapter's
+`ChannelDO`). Fanout is computed per message:
+
+1. The reducer emits `Broadcast("#chan", [line], except=sender)`.
+2. `AwsRuntime.broadcast()` issues a `Query` on `ChannelMembers` with
+   `PK = "#chan"` → returns one row per member.
+3. For each member, the Lambda calls
+   `ApiGatewayManagementApi.postToConnection(member, line)`.
+
+That's O(members) Management API calls per message. A 100-member
+channel receiving one PRIVMSG per second produces ~100 `postToConnection`
+calls/second and one `Query`/second. A 1k-member announcement channel
+at the same rate produces ~1k calls/second. Mitigations (batched
+postToConnection, per-channel send-list cache, a dedicated fanout
+Lambda) are deferred to post-v1; the load-test ticket will surface
+the ceiling.
+
+### 9.4 Removal policy (IMPORTANT)
+
+The stack sets `RemovalPolicy.DESTROY` on every table:
+
+```ts
+// apps/aws-stack/src/aws-stack.ts:48
+const tables = Object.entries(TABLE_DEFS).map(
+  ([logicalId, tableProps]) =>
+    new Table(this, logicalId, { ...tableProps, removalPolicy: RemovalPolicy.DESTROY }),
+);
+```
+
+**This means `cdk destroy` deletes every table and all of its data.**
+This is intentional for staging (you want a clean teardown between
+experiments) and **dangerous for production**.
+
+For production, fork the stack to use `RemovalPolicy.RETAIN` (or
+`RETAIN_ON_UPDATE`): tables become orphans on stack deletion and keep
+their data. There is no built-in knob for this today — it requires
+editing `aws-stack.ts`. Recommendation: ship a `production: boolean`
+stack prop in a future change that flips the removal policy. Flagged
+in §14.
+
+### 9.5 TTL on `Connections.idleSince`
+
+The `Connections` table carries a DynamoDB TTL attribute (`idleSince`),
+asserted at synth time (`stack.test.ts:52`). The Lambda updates
+`idleSince` on every frame; DynamoDB silently deletes rows whose
+`idleSince` is older than the TTL window (typically within 48 hours of
+expiry, no SLA). This is a **safety net** for rows the sweeper misses.
+
+### 9.6 The gone-connection sweeper
+
+A second Lambda (`IrcSweeper`, handler export `sweeperHandler`) runs on
+an EventBridge schedule at `rate(5 minutes)`:
+
+```ts
+new Rule(this, 'SweeperSchedule', {
+  schedule: Schedule.rate(Duration.minutes(5)),
+  targets: [new LambdaFunction(sweeper)],
+});
+```
+
+Each tick `Scan`s the entire `Connections` table, identifies rows past
+a hard stale threshold, and runs the same cleanup path as
+`$disconnect` (delete memberships, release nicks, delete the row). The
+sweeper catches connections that vanished without APIGW emitting
+`$disconnect` (e.g. TCP RST that APIGW didn't observe).
+
+**Cost note:** `Scan` reads the **entire** table every 5 minutes. At
+1 KB per row, scanning 10k connections costs ~3 RRU (10k / 4 KB per
+RRU) per tick, ~864 ticks/day, ~2.6k RRU/day — negligible. At 100k
+connections it's ~26k RRU/day — still small but visible. A future
+optimization is to make the sweeper `Query` a sparse GSI keyed by
+`idleSince` instead of `Scan`.
+
+### 9.7 When to consider provisioned capacity
+
+Switch a table to `PROVISIONED` billing (with or without autoscaling)
+when its traffic is predictable and steady. The clearest signal is
+when the on-demand bill for a single table exceeds the cost of
+provisioned capacity for the same throughput — DynamoDB's pricing page
+breaks this out. In practice:
+
+- `ChannelMembers` is the first candidate (channel fanout dominates).
+- `Connections` is a poor candidate (spiky, cold-start-driven).
+- `Accounts`/`ChannelMeta`/`Nicks` are unlikely to need provisioned
+  capacity in any v1-size deployment.
+
+Switching billing modes is a no-downtime `UpdateTable` operation; the
+stack change is `billingMode: BillingMode.PROVISIONED` plus
+`readCapacity`/`writeCapacity` props.
+
+---
+
+## 10. Region strategy
+
+### 10.1 Everything is regional
+
+API Gateway WebSocket endpoints are regional. DynamoDB is regional.
+EventBridge Scheduler is regional. Lambda is regional. The stack
+deploys into exactly one region (the one your credentials resolve to
+via `CDK_DEFAULT_REGION`).
+
+WebSocket clients pay round-trip time to that region. A user in
+Frankfurt connecting to a `us-east-1` deployment will see ~90 ms RTT
+on every PRIVMSG roundtrip; the same user against `eu-central-1` will
+see ~10 ms. **Pick the region closest to your user base.**
+
+### 10.2 Recommendation: single-region for v1
+
+For v1, deploy in one region. There are no static assets (CloudFront
+and S3 are not part of this stack), so a CDN adds no value here.
+Single-region keeps the data model simple: the cross-table
+`TransactWriteItems` flows (§9.2) only work within one region.
+
+### 10.3 Multi-region is out of scope for v1
+
+If you ever need multi-region active-active, the considerations are:
+
+- **DynamoDB global tables** — replicate all five tables. Eventual
+  consistency across regions complicates the conditional-write
+  invariants (nick uniqueness, membership transactions): a nick
+  reserved in region A may not be visible in region B for a second or
+  two, allowing duplicate reservations. The reducer logic would need
+  to tolerate this.
+- **Per-region APIGW + Lambda** — each region runs the full stack;
+  clients are routed by DNS (Route 53 latency-based routing).
+- **Session affinity** — a client's `connectionId` is regional. A
+  reconnect after the 2-hour APIGW cap must land on the same region
+  or treat the new connection as a fresh registration (re-NICK,
+  re-JOIN).
+
+This is post-v1 work; the AWS adapter architecture doc calls it out
+as an open question.
+
+### 10.4 Region pick shortlist
+
+For a globally distributed user base, the typical picks:
+
+- **US-East-1 (N. Virginia)** — newest services, cheapest, ~100 ms to
+  EU and ~150 ms to APAC. Default for the CI workflow's `AWS_REGION`.
+- **EU-West-1 (Ireland)** / **EU-Central-1 (Frankfurt)** — best for
+  EU users.
+- **AP-Northeast-1 (Tokyo)** / **AP-Southeast-1 (Singapore)** — best
+  for APAC users.
+
+---
+
+## 11. Cost notes
+
+All figures are the public AWS list price as of late 2024 — **always
+verify against the current AWS pricing page** before budgeting:
+
+- API Gateway WebSocket: <https://aws.amazon.com/apigateway/pricing/>
+- Lambda: <https://aws.amazon.com/lambda/pricing/>
+- DynamoDB: <https://aws.amazon.com/dynamodb/pricing/>
+- EventBridge: <https://aws.amazon.com/eventbridge/pricing/>
+
+### 11.1 Billable services
+
+| Service                  | Unit                                           | List price (late 2024)        |
+|--------------------------|------------------------------------------------|-------------------------------|
+| API Gateway WebSocket    | connection-minute                              | $0.80 / million               |
+| API Gateway WebSocket    | message (inbound or outbound)                  | $1.00 / million               |
+| Lambda                   | request                                        | $0.20 / million               |
+| Lambda                   | GB-second                                      | $0.0000166667 (≈ $0.0167 / GB-hour) |
+| DynamoDB on-demand       | write request unit (WRU)                       | $1.25 / million               |
+| DynamoDB on-demand       | read request unit (RRU)                        | $0.25 / million               |
+| EventBridge Scheduler    | event published                                | $1.00 / million               |
+| CloudWatch Logs          | GB ingested                                    | $0.50                         |
+
+The Lambda free tier (1M requests + 400k GB-seconds per month) covers
+a meaningful slice of a small staging deployment.
+
+### 11.2 Back-of-envelope: 100 concurrent users
+
+Assume a small staging deployment with 100 concurrent users, average
+5 messages/sec across all channels, each connection active for 1 hour,
+average Lambda invocation 50 ms at 128 MB:
+
+| Dimension                                | Estimate                              | Monthly cost (approx) |
+|------------------------------------------|---------------------------------------|-----------------------|
+| APIGW connection-minutes                 | 100 conn × 60 min × 24 h × 30 d = 4.32M conn-min | ~$3.50        |
+| APIGW messages (inbound + fanout)        | 5 msg/s × 86400 × 30 = 13M inbound, ~3× fanout | ~$50          |
+| Lambda invocations                       | ~1 per inbound + fanout ≈ 40M         | ~$8.00 (free tier offsets some) |
+| Lambda GB-seconds                        | 40M × 0.05 s × 0.125 GB = 250k GB-s   | ~$4.20                |
+| DynamoDB writes (Connections updates + ChannelMembers deltas) | ~5M WRU | ~$6.25                |
+| DynamoDB reads (fanout Queries + Connections GetItem) | ~30M RRU                  | ~$7.50                |
+| EventBridge events (sweeper + pingChecker) | 5-min + 1-min ticks ≈ 50k events    | <$0.10                |
+| CloudWatch Logs                          | ~5 GB                                 | ~$2.50                |
+
+**Rough total: ~$80/month** for a 100-concurrent-user staging
+deployment. The dominant cost is **APIGW messages** (inbound + the
+per-member fanout). The single biggest lever is channel size — a
+single 1k-member announcement channel can multiply the fanout cost by
+10×.
+
+### 11.3 Cost-trim levers
+
+- **Cap channel size** with `+l` mode (the existing MODE enforcement).
+- **Consider provisioned capacity** for `ChannelMembers` once traffic
+  is steady (§9.7).
+- **Shorten the sweeper cadence** if you can tolerate slower cleanup —
+  `rate(5 minutes)` is a balance; `rate(15 minutes)` cuts sweeper
+  cost 3× at the cost of slower gone-connection reaping.
+- **Tune Lambda memory** — 128 MB is the default and cheapest per
+  invocation, but a higher memory allocation can reduce duration
+  enough to win on GB-seconds.
+
+---
+
+## 12. CI/CD
+
+The GitHub Actions workflow at `.github/workflows/deploy-aws.yml`
+deploys staging on every push to `main` and replays the smoke e2e. It
+is the canonical deploy path — manual `pnpm deploy:aws:staging` is for
+iteration only.
+
+### 12.1 What the workflow does
+
+Steps performed (in order):
+
+1. Checkout.
+2. Install pnpm 9 + Node 20 (`cache: pnpm`).
+3. `pnpm install --frozen-lockfile`.
+4. `pnpm build` — builds all workspace packages.
+5. `pnpm typecheck` and `pnpm test` — gate the deploy.
+6. Configure AWS credentials (OIDC if `AWS_DEPLOY_ROLE_ARN` is set,
+   otherwise static keys).
+7. `pnpm deploy:aws:staging` → `cdk deploy --all --require-approval never`.
+8. Resolve the smoke URL: repo variable `AWS_SMOKE_URL` first, else
+   the stack's `ConnectUrl` CloudFormation output via
+   `aws cloudformation describe-stacks --stack-name IrcAwsStack`, else
+   skip the smoke step with a warning.
+9. `pnpm smoke:aws:staging -- --url "$SMOKE_URL"` — failure fails the
+   build.
+
+Concurrency is serialized via `concurrency.group: aws-staging`,
+`cancel-in-progress: false`, so two pushes cannot race the same
+CloudFormation stack.
+
+### 12.2 Required repo configuration
+
+Under **Settings → Secrets and variables → Actions**:
+
+- **Secret `AWS_ACCESS_KEY_ID`** — required unless OIDC.
+- **Secret `AWS_SECRET_ACCESS_KEY`** — required unless OIDC.
+- **Secret `AWS_DEPLOY_ROLE_ARN`** — optional; enables OIDC.
+- **Secret `AWS_REGION`** — required (e.g. `us-east-1`).
+- **Variable `AWS_SMOKE_URL`** — optional but recommended; the
+  workflow can fall back to the CFN output, but setting this variable
+  explicitly avoids the `describe-stacks` round-trip and the
+  possibility of a stale cached value after a redeploy.
+
+### 12.3 OIDC vs static keys
+
+The workflow supports both auth modes:
+
+```yaml
+- name: Configure AWS credentials
+  uses: aws-actions/configure-aws-credentials@v4
+  with:
+    aws-region: ${{ secrets.AWS_REGION }}
+    role-to-assume: ${{ secrets.AWS_DEPLOY_ROLE_ARN }}
+    aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+    aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+```
+
+When `AWS_DEPLOY_ROLE_ARN` is set, the action assumes the role via
+OIDC web identity (requires `permissions.id-token: write`, which the
+workflow declares) and ignores the static keys. When unset, it falls
+back to the static access-key pair. OIDC is the recommended path for
+any deployment tied to a real AWS account — no long-lived credentials
+to rotate.
+
+### 12.4 No CI auto-deploy to production
+
+Production deploys are intentionally manual (`pnpm deploy:aws:prod`
+from a clean checkout on `main`, against separate credentials). The
+workflow's `on:` block only triggers on `push` to `main` and
+`workflow_dispatch`, both of which target staging.
+
+---
+
+## 13. Troubleshooting
+
+### 13.1 `cdk deploy` fails with bucket-not-found
+
+> `BUCKET_NAME_PARAMETER_<id> does not exist` or
+> `NoSuchBucket: The specified bucket does not exist`
+
+The CDK bootstrap stack hasn't been created in this account/region.
+
+```bash
+cd apps/aws-stack
+pnpm cdk bootstrap
+pnpm deploy:aws:staging
+```
+
+Bootstrap is idempotent and per-region — re-run it if you change
+regions.
+
+### 13.2 Lambda throws on cold start because of a missing env var
+
+Symptom: smoke e2e times out waiting for `001`; CloudWatch Logs for
+`IrcHandler` show an `Error: Missing env var CONNECTIONS_TABLE (table
+name for Connections)` (or one of the other four `<NAME>_TABLE` vars).
+
+The `tablesConfigFromEnv` helper throws on the first invocation if any
+of `CONNECTIONS_TABLE`, `CHANNELMETA_TABLE`, `CHANNELMEMBERS_TABLE`,
+`NICKS_TABLE`, `ACCOUNTS_TABLE` is missing. The CDK stack always
+injects these — if one is missing in the console, somebody hand-edited
+the Lambda config. Fix by re-deploying:
+
+```bash
+pnpm deploy:aws:staging
+```
+
+If the error mentions `MANAGEMENT_URL` being absent: that env var is
+optional at the loader level (`buildDepsFromEnv` tolerates its absence
+by setting `managementApi = null`), but the handler's fanout path will
+fail at the first `postToConnection`. Verify both `IrcHandler` and
+`IrcPingChecker` carry `MANAGEMENT_URL` in the console.
+
+### 13.3 Smoke e2e times out waiting for `001`
+
+The connection upgrade succeeded (the WebSocket `open` fired) but
+registration didn't complete. Common causes:
+
+- **The handler threw on the first frame.** Tail the logs:
+  ```bash
+  aws logs tail /aws/lambda/IrcHandler --follow
+  ```
+  and reproduce. Look for the cold-start `buildDepsFromEnv` throw
+  (§13.2) or an unexpected exception in `handleDefault`.
+- **The `prod` stage didn't deploy.** The stage is `autoDeploy: true`,
+  so this is rare, but verify the URL ends in `/prod` and that the
+  route is wired:
+  ```bash
+  aws apigatewayv2 get-routes --api-id <api-id>
+  ```
+  You should see three routes: `$connect`, `$disconnect`, `$default`.
+- **Client connected over plain `ws://`.** APIGW WebSocket stages only
+  accept `wss://`. Use the URL printed by `cdk deploy` (the
+  `ConnectUrl` output).
+
+### 13.4 `aws cloudformation describe-stacks` returns nothing in CI
+
+The workflow uses:
+
+```bash
+aws cloudformation describe-stacks \
+  --stack-name IrcAwsStack \
+  --query 'Stacks[0].Outputs[?OutputKey==`ConnectUrl`].OutputValue' \
+  --output text
+```
+
+to resolve the smoke URL. If this returns nothing, either:
+
+- **The stack name is wrong.** It's hardcoded as `IrcAwsStack` in
+  `apps/aws-stack/bin/aws.ts`. If you forked the stack to a different
+  name, the workflow's `--stack-name IrcAwsStack` won't find it.
+  Update the workflow or rename back.
+- **The deploy failed silently.** Check the prior `pnpm
+  deploy:aws:staging` step's exit code. `cdk deploy` with
+  `--require-approval never` can still fail on IAM or
+  CloudFormation errors.
+- **Wrong region.** The `describe-stacks` call uses `AWS_REGION` from
+  the secrets; it must match the region the stack was deployed into.
+
+The workflow prints a `::warning::` and skips the smoke step rather
+than failing the build — a silent skip is the failure mode to watch
+for.
+
+### 13.5 WebSocket client can't connect
+
+The `$connect` route is wired to `IrcHandler` and currently uses **no
+auth** (no authorizer). If the client gets a 4xx/5xx on the HTTP
+upgrade:
+
+- Verify the URL is the `ConnectUrl` output, not `ManagementUrl` (the
+  management endpoint rejects WebSocket upgrades).
+- Verify the `IrcHandler` Lambda has `execute-api:ManageConnections`
+  on this stage (the stack grants this via
+  `stage.grantManagementApiAccess(handler)`).
+- If you add an authorizer or IP allowlist later, the `$connect`
+  Lambda is the natural place to gate by source IP (read
+  `event.requestContext.connectionId` and
+  `event.requestContext.sourceIp`).
+
+### 13.6 DynamoDB table vanished after `cdk destroy`
+
+By design. The stack uses `RemovalPolicy.DESTROY` on every table
+(§9.4) so staging teardowns are clean. **Do not run
+`cdk destroy` against a production stack.** For production, fork the
+stack to `RemovalPolicy.RETAIN` before the first deploy; flagged in
+§14.
+
+### 13.7 High DynamoDB cost on `Connections`
+
+The gone-connection sweeper `Scan`s the entire `Connections` table
+every 5 minutes (§9.6). At very large scale this dominates the read
+bill. Short-term mitigation: increase the sweeper's `rate(...)` to
+`rate(15 minutes)` or `rate(30 minutes)` in `aws-stack.ts`. Long-term:
+add a sparse GSI keyed by `idleSince` and switch the sweeper to a
+`Query` against it (post-v1 work).
+
+### 13.8 Nick reservation always fails
+
+A client's `NICK` always returns `433 ERR_NICKNAMEINUSE`, even for
+nicks that should be free. Two deployments sharing the same DynamoDB
+table name space is the usual cause — e.g. you deployed `IrcAwsStack`
+twice into the same account/region with a forked stack name. The
+`Nicks` table name is hardcoded as `Nicks` (physical name), so two
+stacks in the same region collide on the same table. Use separate
+regions or fork the table-name prefix.
+
+---
+
+## 14. Quick reference
+
+```bash
+# One-time account setup (per region)
+aws configure                                     # or: export AWS_PROFILE=...
+cd apps/aws-stack && pnpm cdk bootstrap
+
+# Local dev
+pnpm install
+pnpm build
+pnpm --filter @serverless-ircd/aws-stack test     # synth-time assertions
+pnpm --filter @serverless-ircd/aws-stack cdk:synth # writes cdk.out/
+
+# Optional localstack validation (Docker required)
+docker run --rm -d -p 4566:4566 localstack/localstack
+npm install -g aws-cdk-local
+export AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test AWS_DEFAULT_REGION=us-east-1
+LOCALSTACK=1 pnpm --filter @serverless-ircd/aws-stack test
+
+# Staging
+pnpm deploy:aws:staging
+node apps/aws-stack/scripts/smoke.mjs \
+  --url wss://<id>.execute-api.<region>.amazonaws.com/prod
+aws logs tail /aws/lambda/IrcHandler --follow     # live logs
+
+# Production (manual, separate credentials)
+pnpm deploy:aws:prod
+
+# Tests / lint
+pnpm test                                         # full workspace
+pnpm --filter @serverless-ircd/aws-adapter test
+pnpm --filter @serverless-ircd/aws-stack test
+pnpm lint
+```
+
+Key files:
+
+| Path                                              | What                                                |
+|---------------------------------------------------|-----------------------------------------------------|
+| `apps/aws-stack/src/aws-stack.ts`                 | CDK stack — all constructs and env-var wiring.      |
+| `apps/aws-stack/bin/aws.ts`                       | App entry. Hardcodes stack name `IrcAwsStack`.      |
+| `apps/aws-stack/cdk.json`                         | `app = tsx bin/aws.ts`.                              |
+| `apps/aws-stack/scripts/smoke.mjs`                | CI + local smoke e2e.                               |
+| `apps/aws-stack/tests/stack.test.ts`              | Synth-time assertions (cross-check for every fact). |
+| `packages/aws-adapter/src/cdk-table-defs.ts`      | Five DynamoDB table shapes (keys, billing, TTL).    |
+| `packages/aws-adapter/src/tables.ts`              | Runtime table-name + key-column constants.          |
+| `packages/aws-adapter/src/config-loader.ts`       | Lambda env → `ServerConfig` schema mapping.         |
+| `packages/aws-adapter/src/handlers/index.ts`      | Lambda entry points (`handler`, `sweeperHandler`, `pingCheckerHandler`). |
+| `.github/workflows/deploy-aws.yml`                | Staging deploy + smoke e2e CI.                      |
+
+### Known gaps flagged for follow-up
+
+These are facts the **code does not yet have an answer for**; the
+recommendations above are deployer guidance, not built-in knobs:
+
+1. **`RemovalPolicy.DESTROY` is hardcoded.** There is no stack prop to
+   flip it to `RETAIN` for production. Recommendation: a future change
+   adds a `production: boolean` (or similar) construct prop that
+   toggles the removal policy across all five tables.
+2. ~~**`SERVER_NAME` / `NETWORK_NAME` / `MOTD` are hardcoded in
+   `aws-stack.ts`.**~~ **Resolved** — these are now CDK construct props
+   (`serverName`, `networkName`, `motdLines`) on `IrcStackProps` (see
+   §7.1).
+3. ~~**MOTD multiline parsing.**~~ **Resolved** — the stack's
+   `motdLines` prop is a `string[]` joined with `\n`, which the config
+   loader splits back into one line per entry. The delimiter is `\n`
+   only; commas are treated as literal line content (see §7.1).
+4. **No per-environment stack parameter.** Staging and production
+   cannot coexist in the same account/region without forking the
+   stack to parameterize names. Recommendation: separate accounts for
+   staging vs production for v1.