serverless-ircd 0.5.0 → 0.7.0

package/docs/AWS-TCP-Deployment.md

@@ -0,0 +1,410 @@
+# AWS TCP+TLS Deployment Guide (`irc+tls` :6697)
+
+End-to-end guide for deploying the **raw TCP+TLS** transport of
+ServerlessIRCd to AWS (Phase 7 of `PLAN.md`, TICKET-056). This is the
+RFC 1459/2812-compliant path that older clients, bots, and bridges use.
+The default **wss** transport is documented separately in
+[`AWS-Deployment.md`](./AWS-Deployment.md); the two coexist in a single
+CDK stack and a deployment can enable either or both.
+
+> **Read first:** [`ADR-009`](./ADR-009-dual-transport-wss-and-irc-tls.md)
+> for *why* a second transport exists and the per-platform mapping.
+> This doc covers the *how*.
+
+**Acceptance criterion (TICKET-058):** a new contributor can deploy
+their own TCP+TLS staging instance following only this doc.
+
+Cross-reference: `apps/aws-stack/src/aws-stack.ts` (CDK), the wss path
+(`AWS-Deployment.md`), TICKET-054 (mTLS), ADR-006 (SASL scope).
+
+---
+
+## 1. What gets deployed
+
+The wss path is API Gateway WebSocket → Lambda → DynamoDB. The TCP path
+**cannot** reuse API Gateway (it is WebSocket-only), so it adds a second
+entry point: a **Network Load Balancer** with a `TLS` listener that
+terminates TLS and invokes a **Lambda streaming function** as the
+target. The Lambda reuses the exact same DynamoDB tables and the same
+IRC core as the wss path — only the I/O shell differs.
+
+```
+  IRC client (irc+tls :6697)
+        │
+        │  TLS handshake (client ⟷ NLB / ACM cert)
+        ▼
+  ┌──────────────────────────────────────────────────────────────┐
+  │  Network Load Balancer  (internet-facing, 2-AZ VPC)          │
+  │    • TLS listener on :6697 (ACM certificate)                 │
+  │    • terminates TLS, forwards plaintext chunks               │
+  │    • flow metadata: x-forwarded-for / x-forwarded-port       │
+  └──────────────────────────┬───────────────────────────────────┘
+                             │ invoke (per chunk, base64 body)
+                             ▼
+  ┌──────────────────────────────────────────────────────────────┐
+  │  Lambda: IrcNlbHandler  (Node 24, nlbStreamHandler)          │
+  │    • TcpByteStreamTransport reassembles \r\n lines           │
+  │      (partial-line buffer persisted to Connections row)      │
+  │    • ConnectionActor + AwsRuntime (same core as $default)    │
+  │    • connId = nlb-<ip>-<port> (distinct from APIGW ids)     │
+    └──────────────────────────┬───────────────────────────────────┘
+                             │ read/write
+                             ▼
+  ┌──────────────────────────────────────────────────────────────┐
+  │  DynamoDB (same 5 tables as the wss path)                    │
+  │    Connections (+ transportBuffer attr)  ChannelMeta         │
+  │    ChannelMembers   Nicks   Accounts                          │
+  └──────────────────────────────────────────────────────────────┘
+```
+
+Both transports live in **one CDK stack** (`IrcAwsStack`). The TCP path
+is opt-in: it is only provisioned when `tcpTlsDomainName` is supplied.
+When enabled, the stack adds (verified by `apps/aws-stack/tests/stack.test.ts`):
+
+| Resource                              | Purpose                                            |
+|---------------------------------------|----------------------------------------------------|
+| `AWS::CertificateManager::Certificate`| TLS cert for the domain (DNS-validated).           |
+| `AWS::EC2::Vpc` (`NlbVpc`)            | Dedicated 2-AZ VPC, **no NAT gateways**.           |
+| `AWS::ElasticLoadBalancingV2::LoadBalancer` | Internet-facing NLB.                        |
+| `AWS::ElasticLoadBalancingV2::Listener` (`IrcNlbTlsListener`) | `TLS` on :6697.        |
+| `AWS::ElasticLoadBalancingV2::TargetGroup` | `TargetType: lambda`, target = `IrcNlbHandler`. |
+| `AWS::Lambda::Function` (`IrcNlbHandler`) | `nlbStreamHandler` export, Node 24.           |
+| `CfnOutput: TcpConnectUrl`            | `<nlb-dns>:6697`.                                  |
+
+The **parser → reducer → dispatch pipeline is shared verbatim** with the
+wss path (`$default`). Only the framing differs: each NLB chunk is a
+fresh Lambda invocation, so the `TcpByteStreamTransport`'s partial-line
+buffer is snapshotted to a `transportBuffer` attribute on the
+`Connections` row and rehydrated on the next chunk — no bytes are lost
+across the compute boundary.
+
+---
+
+## 2. Prerequisites
+
+| Requirement       | Detail                                                              |
+|-------------------|---------------------------------------------------------------------|
+| Everything in `AWS-Deployment.md` §2 | Same CDK v2, pnpm 11, Node 24, AWS account, `cdk bootstrap`. |
+| ACM certificate   | Provisioned automatically by the stack for the domain you pass; the hosted zone must exist in Route 53 (or be validated out-of-band) for DNS validation. |
+| A domain          | The public hostname (e.g. `irc.example.com`) you will pass as `tcpTlsDomainName`. |
+| NLB Lambda invoke | The target group uses `TargetType.LAMBDA`; no ALB/Lambda-HTTP needed. |
+
+No extra IAM beyond what the stack already grants — the CDK adds the
+`elasticloadbalancing.amazonaws.com` invoke permission and the NLB
+handler's DynamoDB grants automatically.
+
+---
+
+## 3. How the TCP path is enabled
+
+The `IrcAwsStack` construct accepts an optional prop
+(`apps/aws-stack/src/aws-stack.ts`):
+
+```ts
+export interface IrcStackProps extends StackProps {
+  readonly environmentName?: string;
+  readonly serverName?: string;
+  readonly networkName?: string;
+  readonly motdLines?: string[];
+  /** When provided, provisions NLB + ACM cert + Lambda streaming target. */
+  readonly tcpTlsDomainName?: string;
+}
+```
+
+Omit `tcpTlsDomainName` → only the wss (API Gateway WebSocket) path is
+deployed (the default; what `AWS-Deployment.md` describes). Provide it →
+the TCP+TLS path is **added alongside** the wss path; both share the
+same DynamoDB tables, server identity, and `MANAGEMENT_URL`.
+
+`bin/aws.ts` reads `tcpTlsDomainName` from CDK context so it can be
+passed on the CLI without editing source:
+
+```ts
+const tcpTlsDomainName =
+  app.node.tryGetContext('tcpTlsDomainName') as string | undefined;
+```
+
+…then forwarded into `IrcStackProps.tcpTlsDomainName`. When the context
+var is absent (the default), the prop is omitted and only the wss path
+deploys — the existing pipeline stays green.
+
+---
+
+## 4. Deploy staging (TCP+TLS)
+
+```bash
+# From the repo root:
+pnpm deploy:aws:staging -- \
+  -c environmentName=staging \
+  -c tcpTlsDomainName=irc-staging.example.com
+```
+
+This is the same `cdk deploy --all` the wss path uses
+(`apps/aws-stack/package.json` `deploy:staging`), just with the extra
+context var. The deploy will:
+
+1. Create the ACM certificate for `irc-staging.example.com` (DNS
+   validation via Route 53; if the hosted zone is not in this account,
+   validate out-of-band and import the cert ARN — see §9.2).
+2. Provision `NlbVpc`, the internet-facing NLB, the `TLS` listener on
+   `:6697`, and the `IrcNlbHandler` Lambda target group.
+3. Grant the NLB handler the same DynamoDB access as the wss handler
+   and inject the same table names, `SERVER_NAME`, `NETWORK_NAME`,
+   `MOTD`, and `MANAGEMENT_URL`.
+4. Emit the new stack output:
+
+```
+TcpConnectUrl = <nlb-dns>.elb.<region>.amazonaws.com:6697
+```
+
+The wss path (`ConnectUrl`, `ManagementUrl`) is **still deployed and
+unchanged** — enabling TCP does not disable wss.
+
+### 4.1 Wait for DNS / cert validation
+
+The first deploy can take several minutes while ACM validates the
+certificate. `cdk deploy` waits on the certificate resource; do not
+interrupt it. If validation times out, confirm the hosted zone for the
+domain is in this account and Route 53 can write the `_acme-challenge`
+CNAME.
+
+### 4.2 Point a DNS record at the NLB (recommended)
+
+The NLB prints a long `.elb.amazonaws.com` DNS name. For a stable,
+owned hostname, create an **alias A record** (Route 53) or a CNAME
+pointing `irc-staging.example.com` → the NLB DNS name. Clients then
+connect to `irc-staging.example.com:6697`.
+
+(You can connect directly to the raw NLB DNS, but the cert is issued for
+your domain, so TLS SNI/hostname verification wants the owned name.)
+
+### 4.3 Smoke test over TLS
+
+```bash
+printf 'NICK smoke\r\nUSER smoke 0 * :Smoke\r\nQUIT :smoke\r\n' | \
+  timeout 10 openssl s_client -connect irc-staging.example.com:6697 -quiet | \
+  grep -q '001' && echo "SMOKE PASS" || echo "SMOKE FAIL"
+```
+
+---
+
+## 5. Deploy production
+
+```bash
+pnpm deploy:aws:prod -- \
+  -c environmentName=production \
+  -c tcpTlsDomainName=irc.example.com
+```
+
+Production deploys are manual. The staging CI workflow
+(`deploy-aws.yml`) deploys only the wss path (`-c environmentName=staging`
+without `tcpTlsDomainName`); there is **no auto-deploy for the TCP path
+yet** — add a `deploy-aws-tcp.yml` workflow (mirroring `deploy-cf-tcp.yml`)
+as a follow-up when the TCP path is ready for continuous staging.
+
+---
+
+## 6. Operational model & limits
+
+The NLB + Lambda streaming model has hard platform limits the wss path
+does not. Understanding these is **required** before production:
+
+| Limit                          | Impact on IRC                                              |
+|--------------------------------|------------------------------------------------------------|
+| **NLB idle timeout** (10–350s) | A connection with no client data for this long is closed. IRC clients that only receive (lurkers) hit this. Mitigation: client-side PING, or accept reconnects. |
+| **No server-push**             | The Lambda can only respond to a client chunk; it cannot proactively push bytes. Real-time fanout (PRIVMSG broadcast, QUIT) **piggybacks on the recipient's next chunk** (response) or flows via the `MANAGEMENT_URL` (APIGW `postToConnection`) when both transports are deployed. |
+| **No force-close from Lambda** | A `Disconnect("Excess Flood")` effect cannot close the NLB-held socket from inside the invocation. Canonical teardown is the sweeper + `idleSince` TTL (same pattern as the APIGW `$default` path). |
+| **Flow rebalancing**           | The NLB may rebalance the underlying flow at any time; the 4-tuple `(ip, port)` stays stable for the connection's life, so `connId` (`nlb-<ip>-<port>`) is stable. |
+| **Lambda invoke per chunk**    | Each client byte-chunk is one invocation. The `transportBuffer` reassembly across invocations is correct but adds a DynamoDB write per chunk. Coalescing client writes reduces cost. |
+
+Because of the no-server-push limit, **deploying both transports is the
+recommended shape for full-featured IRC**: the wss path (or just the
+`MANAGEMENT_URL` from it) provides the push channel for real-time
+fanout, while the TCP path serves RFC-compliant clients. An NLB-only
+deploy (`managementApi = null`) silently no-ops cross-connection sends —
+documented platform limit, see `packages/aws-adapter/src/handlers/nlb-stream.ts`.
+
+State lifecycle is identical to the wss path: the **sweeper** Lambda
+(`rate(5 minutes)`) prunes `Connections` rows past the hard TTL, and the
+**ping-checker** Lambda (`rate(1 minute)`) sends PING / disconnects
+no-PONG. Both cover TCP connections because the `connId` lives in the
+same `Connections` table.
+
+---
+
+## 7. Configuration reference
+
+All knobs are CDK context vars (`-c key=value`) or construct props. The
+TCP-specific ones:
+
+| Knob (context var / prop)     | Purpose                                          | Default when omitted       |
+|-------------------------------|--------------------------------------------------|----------------------------|
+| `tcpTlsDomainName`            | Domain for the ACM cert + NLB TLS listener.      | *(TCP path not provisioned)* |
+| `environmentName`             | Env prefix for table names + stack id.           | `staging`                  |
+
+The NLB handler Lambda inherits the **same** identity/table knobs as the
+wss handler (`serverName`, `networkName`, `motd`, `environmentName`),
+injected in `aws-stack.ts`:
+
+- `SERVER_NAME`, `NETWORK_NAME`, `MOTD` (read by the shared
+  `config-loader.ts`).
+- `MANAGEMENT_URL` — the wss stage's callback URL, so cross-connection
+  fanout reaches recipients on either transport.
+- `<TABLE>_TABLE` for each of the five tables (prefixed by environment).
+
+The Lambda handler entry is the same file
+(`packages/aws-adapter/src/handlers/index.ts`); esbuild tree-shakes to
+the `nlbStreamHandler` export. Runtime `nodejs24.x`; `@aws-sdk/*`
+excluded from the bundle (ships with the runtime).
+
+There are **no TCP-specific DynamoDB tables** — the existing `Connections`
+table gains a `transportBuffer` string attribute (SET when non-empty,
+REMOVE when empty) on rows whose `connectionId` starts with `nlb-`.
+
+---
+
+## 8. mTLS (SASL EXTERNAL)
+
+SASL `EXTERNAL` needs the client certificate to reach the application
+(ADR-006, TICKET-054). On the TCP path:
+
+- The NLB `TLS` listener terminates TLS with the ACM cert.
+- For client-cert (mTLS) verification, configure the NLB listener's
+  **mutual TLS / trust store** (AWS: a trust-store bundle of CA certs on
+  the listener). The verified client-cert subject is then surfaced and
+  the origin's `MtlsIdentityProvider` port resolves it to an account,
+  enabling `AUTHENTICATE EXTERNAL` (`903` on success, `904` otherwise).
+- Without a trust store on the listener, EXTERNAL is not available and
+  the `sasl` cap advertises `PLAIN` only (driven by whether
+  `ctx.mtlsIdentity` is defined — TICKET-079).
+
+Cross-reference: TICKET-054 (mTLS support), TICKET-079 (EXTERNAL
+advertisement). The wss path achieves the same via API Gateway
+custom-domain mTLS (`AWS-Deployment.md`).
+
+---
+
+## 9. Troubleshooting
+
+### 9.1 `cdk deploy` hangs on the ACM certificate
+
+DNS validation is waiting on a CNAME it cannot create. Confirm the
+hosted zone for `tcpTlsDomainName` is in this AWS account/region. If the
+zone is elsewhere, validate out-of-band and import the certificate:
+
+```ts
+// In aws-stack.ts, replace `new Certificate(...)` with:
+import { Certificate, Certificate.fromCertificateArn } from 'aws-cdk-lib/aws-certificatemanager';
+const cert = Certificate.fromCertificateArn(this, 'TlsCert', 'arn:aws:acm:...:certificate/...');
+```
+
+(This is a code edit; the default path assumes in-account Route 53.)
+
+### 9.2 Client connects to :6697 but gets no `001`
+
+TLS completed (NLB + cert are fine) but the Lambda did not respond with
+the welcome block. Causes:
+
+- The first chunk did not contain a complete `NICK`/`USER` line (split
+  across chunks). The `transportBuffer` will complete it on the next
+  chunk — send the rest. A well-behaved client sends the full handshake
+  in one write.
+- CloudWatch Logs for `IrcNlbHandler` show an actor throw. The handler
+  catches and logs (`[nlb-stream] actor.receiveTextFrame failed`); it
+  does **not** crash — the connection continues, but a malformed line is
+  dropped. Tail with `aws logs tail /aws/lambda/<IrcNlbHandler>`.
+- `MANAGEMENT_URL` is wrong → cross-connection fanout no-ops, but
+  self-reply (welcome, PONG) still works, so `001` should still appear.
+
+### 9.3 Broadcasts (PRIVMSG to a channel) don't reach TCP recipients
+
+The no-server-push limit (§6). Outbound bytes only flow when the
+recipient sends a chunk (the Lambda response). For real-time delivery,
+ensure the deploy includes the wss path so `MANAGEMENT_URL` is set —
+then fanout uses `ApiGatewayManagementApi.postToConnection`, which works
+for both transports. If `MANAGEMENT_URL` is unset, `managementApi` is
+`null` and cross-connection sends are silent no-ops.
+
+### 9.4 Lurkers get disconnected after a few minutes
+
+The NLB idle timeout. A receiving-only client sends no chunks, so the
+Lambda is never invoked and the NLB closes the idle flow. Mitigations:
+
+- Configure the client to send a periodic `PING` (most clients do).
+- Raise the NLB idle timeout (max 350s) — but this only delays, it does
+  not eliminate, the issue. The wss path does not have this limit and is
+  preferred for lurkers.
+
+### 9.5 Messages split across chunks get lost
+
+This is a transport-layer regression, not config — the
+`TcpByteStreamTransport` + `transportBuffer` persistence is contract-
+tested (`packages/aws-adapter/tests/nlb-stream.test.ts`). If you see it,
+run `pnpm --filter @serverless-ircd/aws-adapter test`; a failure there
+will pinpoint it.
+
+### 9.6 `TcpConnectUrl` output is missing
+
+`tcpTlsDomainName` was not passed. Confirm `-c tcpTlsDomainName=...` is
+on the `cdk deploy` command and that `bin/aws.ts` forwards it into
+`IrcStackProps` (§3). Without it, only the wss resources deploy.
+
+---
+
+## 10. Cost notes (versus the wss path)
+
+| Dimension              | wss path (APIGW WebSocket)             | TCP path (NLB + Lambda streaming)            |
+|------------------------|----------------------------------------|----------------------------------------------|
+| **Per-message**        | APIGW WebSocket + 1 Lambda invoke.     | **1 Lambda invoke per byte-chunk** (more invokes for chatty clients). |
+| **Idle connections**   | APIGW connection minutes.              | NLB LCUs (Load Balancer Capacity Units) per hour; no Lambda cost when idle. |
+| **TLS termination**    | APIGW (managed).                       | NLB (per-LCU) + ACM (free).                  |
+| **State**              | DynamoDB (shared).                     | DynamoDB (shared) + extra `transportBuffer` write per chunk. |
+| **Cross-connection fanout** | `postToConnection` per recipient. | Same — but only works if `MANAGEMENT_URL` (wss path) is deployed. |
+
+The dominant cost driver on the TCP path is **invocations per chunk**.
+Clients that coalesce writes (send the full `NICK\r\nUSER\r\n` in one
+chunk, not byte-by-byte) are dramatically cheaper. Recommend the wss
+path for high-fanout / lurker-heavy workloads and the TCP path for
+RFC-compliant clients that need it.
+
+---
+
+## 11. Quick reference
+
+```bash
+# Staging (TCP+TLS added alongside wss)
+pnpm deploy:aws:staging -- \
+  -c environmentName=staging \
+  -c tcpTlsDomainName=irc-staging.example.com
+
+# Read the TCP endpoint from stack outputs
+aws cloudformation describe-stacks \
+  --stack-name IrcAwsStack-staging \
+  --query 'Stacks[0].Outputs[?OutputKey==`TcpConnectUrl`].OutputValue' \
+  --output text
+
+# Smoke over TLS
+printf 'NICK smoke\r\nUSER smoke 0 * :Smoke\r\nQUIT :smoke\r\n' | \
+  openssl s_client -connect irc-staging.example.com:6697 -quiet | grep '001'
+
+# Production (manual)
+pnpm deploy:aws:prod -- \
+  -c environmentName=production \
+  -c tcpTlsDomainName=irc.example.com
+
+# Tests (synth-time assertions for the NLB path)
+pnpm --filter @serverless-ircd/aws-stack test
+pnpm --filter @serverless-ircd/aws-adapter test
+```
+
+Key files:
+
+| Path                                              | What                                       |
+|---------------------------------------------------|--------------------------------------------|
+| `apps/aws-stack/src/aws-stack.ts`                 | CDK stack — NLB/VPC/cert conditional block.|
+| `apps/aws-stack/bin/aws.ts`                       | Reads `tcpTlsDomainName` from context.     |
+| `packages/aws-adapter/src/handlers/nlb-stream.ts` | `handleNlbStream` + `deriveNlbConnectionId`.|
+| `packages/aws-adapter/src/handlers/index.ts`      | `nlbStreamHandler` Lambda export.          |
+| `apps/aws-stack/tests/stack.test.ts`              | Synth-time NLB assertions (lines 466–559). |
+| `packages/aws-adapter/tests/nlb-stream.test.ts`   | NLB framing/transport-buffer unit tests.   |
+| `docs/AWS-Deployment.md`                          | The wss path this sits alongside.          |