serverless-ircd 0.1.0 → 0.2.0

package/PLAN.md

@@ -0,0 +1,420 @@
+# ServerlessIRCd — Project Plan
+
+A serverless IRC daemon: the IRC protocol logic is a pure, platform-agnostic
+core; two thin adapters run it on **Cloudflare Workers** (Durable Objects) or
+**AWS** (API Gateway WebSockets + Lambda + DynamoDB).
+
+Single logical server (no S2S linking in v1). Practical modern IRC subset plus
+selected IRCv3 extensions. Production-ish quality, strict TDD.
+
+---
+
+## 1. Goals & non-goals
+
+**Goals**
+- One TypeScript codebase implementing IRC server semantics, testable with zero
+  cloud dependencies.
+- Two deployable adapters (CF Workers, AWS) sharing that core.
+- Works with modern clients (WeeChat, HexChat, IRCCloud, TheLounge).
+- ≥90% coverage on core; 100% on protocol/parsing and command reducers.
+
+**Non-goals (v1)**
+- Server-to-server linking / federated networks (TS6, IRCv3 S2S).
+- Full RFC 2812 legacy compliance.
+- Services (NickServ/ChanServ) — designed for, not built.
+- Operator server-mesh features (connect/jupe/squit routing).
+
+**Design-for-later**
+- A `ServerLink` seam in the core so S2S can be added without rewriting handlers.
+
+---
+
+## 2. Architecture
+
+Hexagonal / ports-and-adapters. The core reasons about IRC; adapters move bytes.
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│  packages/irc-core         (pure TS, no cloud deps)             │
+│    protocol/  parse · serialize · numerics · messages           │
+│    commands/  pure reducers: (state, msg) → { state, effects }  │
+│    state/     channel · connection · mode · registry shapes     │
+│    caps/      IRCv3 capability negotiation table                │
+│    isupport/  005 generation                                    │
+└─────────────────────────────────────────────────────────────────┘
+                               ▲ implements IrcRuntime (port)
+                               │
+┌─────────────────────────────────────────────────────────────────┐
+│  packages/irc-server      (orchestration, defines the port)     │
+│    IrcRuntime   side-effect interface (transport + state IO)    │
+│    ConnectionActor  bytes → parse → reducer → runtime           │
+│    location table   which authority owns each command           │
+└─────────────────────────────────────────────────────────────────┘
+            ▲                                   ▲
+┌────────────────────────────┐      ┌──────────────────────────────┐
+│  adapters/cf               │      │  adapters/aws                │
+│    ConnectionDO            │      │    $connect/$disconnect/     │
+│    ChannelDO               │      │    $default Lambda handlers  │
+│    RegistryDO              │      │    DynamoDB tables           │
+│    CfRuntime               │      │    AwsRuntime                │
+│    wrangler.toml           │      │    CDK stack                 │
+└────────────────────────────┘      └──────────────────────────────┘
+            ▲                                   ▲
+┌─────────────────────────────────────────────────────────────────┐
+│  apps/cf-worker   apps/aws-stack   (thin deploy glue)           │
+└─────────────────────────────────────────────────────────────────┘
+
+  packages/in-memory-runtime  ← reference impl of IrcRuntime, used by
+                                 integration tests and a local CLI server
+```
+
+### 2.1 The key idea: pure reducers + location-of-authority
+
+Each command handler is a **pure function**:
+
+```ts
+type Reducer<S> = (state: S, msg: IrcMessage, ctx: Ctx) => { state: S; effects: Effect[] };
+```
+
+Side effects are values (`Effect[]`), not performed in the handler. This makes
+every handler a trivial unit test: arrange state, apply message, assert state'
+and emitted effects.
+
+**Where does each reducer run?** Whichever authority owns the state it mutates.
+This avoids cross-instance read-modify-write races:
+
+| Command family              | Runs in authority       | Primary state mutated        |
+|-----------------------------|-------------------------|------------------------------|
+| Registration (NICK/USER)    | Connection entity       | Connection state             |
+| PING/PONG, QUIT, AWAY       | Connection entity       | Connection state             |
+| User MODE, SETNAME          | Connection entity       | Connection state             |
+| JOIN / PART / channel MODE  | Channel entity          | Roster, channel modes        |
+| PRIVMSG/NOTICE to channel   | Channel entity          | Fanout (reads roster)        |
+| TOPIC, KICK, INVITE         | Channel entity          | Channel state / roster       |
+| PRIVMSG/NOTICE to user      | Sender Connection       | (uses Nick→Connection route) |
+| Nick collision check        | Registry entity         | Nick→Connection map          |
+| NAMES / WHO / WHOIS         | Reads snapshots         | (no mutation)                |
+
+Cross-authority coordination (e.g., JOIN must update the connection's joined
+list **and** the channel roster) is done by the authority that accepts the
+action emitting an effect the other authority consumes. On CF this is a stub()
+call between DOs; on AWS a `TransactWriteItems` plus EventBridge fanout.
+
+### 2.2 The `IrcRuntime` port (side-effect interface)
+
+The set of capabilities a reducer's emitted effects require. The in-memory,
+CF, and AWS runtimes all implement it:
+
+```ts
+interface IrcRuntime {
+  // transport
+  send(to: ConnId, lines: RawLine[]): Promise<void>;
+  broadcast(chan: ChanName, lines: RawLine[], except?: ConnId): Promise<void>;
+  disconnect(conn: ConnId, reason?: string): Promise<void>;
+
+  // nickname registry
+  reserveNick(nick: Nick, conn: ConnId): Promise<{ ok: true } | { ok: false }>;
+  changeNick(conn: ConnId, oldNick: Nick, newNick: Nick): Promise<boolean>;
+  releaseNick(nick: Nick): Promise<void>;
+
+  // channel ownership
+  applyChannelDelta(chan: ChanName, delta: ChannelDelta): Promise<void>;
+
+  // lookups (NAMES, WHOIS, routing)
+  lookupNick(nick: Nick): Promise<ConnId | null>;
+  getConnectionInfo(conn: ConnId): Promise<ConnSnapshot | null>;
+  getChannelSnapshot(chan: ChanName): Promise<ChanSnapshot | null>;
+}
+```
+
+`Effect[]` values are interpreted by a single `dispatch(effects, runtime)`
+function in `irc-server`. The reducer stays pure; only `dispatch` awaits.
+
+---
+
+## 3. Protocol scope (v1)
+
+**Core (RFC 1459/2812 subset)**
+- Registration: `NICK`, `USER`, `CAP` negotiation, `PASS` (server password)
+- `PING` / `PONG` (server-initiated keepalive + client-initiated)
+- `QUIT`, `JOIN`, `PART`, `PRIVMSG`, `NOTICE`
+- `MODE` (user + channel), `TOPIC`, `KICK`, `INVITE`
+- `NAMES`, `LIST`, `WHO`, `WHOIS`, `MOTD`
+- Numeric replies: `001`–`005`, `311`–`319`, `321`–`323`, `331`/`332`,
+  `341`, `352`/`315`, `353`/`366`, `367`–`369`, `372`/`375`/`376`,
+  `401`–`407`, `421`, `432`–`437`, `441`–`449`, `461`–`473`, `482`,
+  `501`/`502`, errors as needed.
+
+**Channel modes**: `o v b i k l t n m s p` (op, voice, ban, invite-only, key,
+limit, topic-lock, no-external, moderated, secret, private).
+**User modes**: `i` (invisible), `o` (oper — local only), `w`, `s`.
+
+**IRCv3 extensions (negotiated via `CAP`)**
+- `message-tags` (`+tag`), `server-time` (`time`), `account-tag`
+- `echo-message`
+- `batch`
+- `sasl` (mechanisms: `PLAIN`, `EXTERNAL` reserved for later with mTLS)
+- `multi-prefix`
+- `away-notify`, `chghost`
+- `invite-notify`, `extended-join`
+
+**Connection framing**: WebSockets carry one IRC message per text frame
+(ws text frames are IRC-line-delimited; we also tolerate `\r\n`-joined frames).
+No plaintext TCP in v1 (wss only) — keeps serverless story clean.
+
+---
+
+## 4. State model & authoritative ownership
+
+| State                       | Authority (CF / AWS)                        | Keyed by           |
+|-----------------------------|---------------------------------------------|--------------------|
+| Connection (nick,user,hostmask,caps,away,joined list) | ConnectionDO / DynamoDB `Connections` | `connectionId`     |
+| Channel (topic,modes,key,limit,ban masks)             | ChannelDO / DynamoDB `ChannelMeta`    | lowercased name    |
+| Membership (members + prefixes)                        | ChannelDO / DynamoDB `ChannelMembers` | `channelId` (PK), `connectionId` (SK) |
+| Nick registry (nick → connectionId)                    | RegistryDO / DynamoDB `Nicks`         | lowercased nick    |
+| Authenticated account (SASL)                           | Shared secret store / DynamoDB `Accounts` | account name   |
+
+**Invariant**: nickname uniqueness and channel membership are enforced with
+conditional writes (DynamoDB `ConditionExpression`) or DO single-threaded
+serial execution — there is exactly one authority per piece of state, so races
+are structurally impossible rather than defended against.
+
+---
+
+## 5. Repository layout
+
+```
+ServerlessIRCd/
+├── packages/
+│   ├── irc-core/                 pure protocol + reducers (the brain)
+│   │   ├── src/
+│   │   │   ├── protocol/  parser.ts serializer.ts numerics.ts messages.ts
+│   │   │   ├── commands/  registration.ts join.ts part.ts privmsg.ts
+│   │   │   │              mode.ts topic.ts kick.ts ping.ts quit.ts names.ts
+│   │   │   │              who.ts whois.ts cap.ts sasl.ts list.ts motd.ts
+│   │   │   ├── state/     connection.ts channel.ts mode.ts registry.ts
+│   │   │   ├── caps/      capabilities.ts
+│   │   │   ├── isupport/  isupport.ts
+│   │   │   ├── effects.ts types.ts ids.ts clock.ts (ports)
+│   │   │   └── index.ts
+│   │   └── tests/  (unit, golden vectors)
+│   ├── irc-server/               orchestration, IrcRuntime port, dispatch
+│   │   ├── src/{runtime.ts, actor.ts, dispatch.ts, location.ts}
+│   │   └── tests/
+│   ├── in-memory-runtime/        reference runtime + local CLI server
+│   ├── cf-adapter/               Durable Objects + CfRuntime
+│   │   ├── src/{connection-do.ts channel-do.ts registry-do.ts runtime.ts env.ts}
+│   │   ├── tests/  (vitest-pool-workers / Miniflare)
+│   │   └── wrangler.toml
+│   └── aws-adapter/              Lambda + DynamoDB + AwsRuntime
+│       ├── src/{handlers runtime tables.ts}
+│       ├── tests/ (dynamodb-local / localstack)
+│       └── cdk/  (infra stack)
+├── apps/
+│   ├── cf-worker/                worker entry, DO migrations, bindings
+│   ├── aws-stack/                CDK app entry
+│   └── local-cli/                runnable server using in-memory-runtime
+├── tools/
+│   ├── irc-client/               minimal test client for e2e
+│   └── load-test/                synthetic client fan-out
+├── tests/
+│   ├── integration/              protocol scenarios via IrcRuntime contract
+│   ├── e2e/                      real WebSocket clients vs deployed adapter
+│   └── fixtures/                 golden messages, capture scripts
+├── docs/
+│   ├── architecture.md  protocol-scope.md  deployment-cf.md  deployment-aws.md
+│   └── adr/                      architecture decision records
+├── pnpm-workspace.yaml  turbo.json  tsconfig.base.json
+├── biome.json  (or .eslintrc + .prettierrc)  vitest.config.ts
+├── .github/workflows/ci.yml
+└── PLAN.md  AGENTS.md  README.md
+```
+
+---
+
+## 6. Platform mapping
+
+### 6.1 Cloudflare Workers
+
+- **Edge worker**: WebSocket upgrade entry; routes each connection to its
+  `ConnectionDO` (`idFromName(connectionId)`). Cheap hibernatable WS via
+  `HibernatingWebSocketHandler` keeps idle connections free.
+- **`ConnectionDO`**: owns one client's socket + connection state; runs
+  connection-scoped reducers; on QUIT/PART/JOIN issues stub() calls into the
+  relevant `ChannelDO`(s) and `RegistryDO`.
+- **`ChannelDO`** (keyed by lowercased channel): serial execution → authoritative
+  roster + channel modes + fanout. Broadcast = loop members, call each
+  `ConnectionDO.fetch()`/`webSocket.send` via stored routing tag.
+- **`RegistryDO`** (sharded by `hash(nick) % N` for hot-nick scaling): nickname
+  uniqueness + nick→connectionId map. Conditional reserve via single-threaded DO.
+- **Persistence**: `state.storage` (DO transactional storage) for durable state;
+  optional `D1`/`KV` for MOTD, config, accounts; `Durable Object alarms` for
+  PING/idle timeouts.
+- **Tests**: `@cloudflare/vitest-pool-workers` against real workerd.
+
+### 6.2 AWS
+
+- **API Gateway WebSocket API**: `$connect`, `$disconnect`, `$default` → Lambda
+  (Node 20). The `connectionId` is the stable connection identity; send via
+  `ApiGatewayManagementApi.postToConnection`.
+- **DynamoDB**:
+  - `Connections` (PK `connectionId`): nick, user, hostmask, caps, joined
+    channels, idle since, callback endpoint.
+  - `ChannelMeta` (PK `channelName`): topic, modes, key, limit.
+  - `ChannelMembers` (PK `channelName`, SK `connectionId`): prefixes.
+  - `Nicks` (PK `nickLower`): `connectionId` (unique via conditional PutItem).
+  - `Accounts` (PK `account`): SASL credentials (never plaintext — store hash).
+- **Transactions**: `TransactWriteItems` for membership changes that span
+  Connections ↔ ChannelMembers.
+- **Fanout**: `$default` Lambda loads the channel's members (query), then
+  `postToConnection` per member; failures (GoneException) mark the connection
+  disconnected and trigger cleanup (lazy on next touch + a sweeper).
+- **Timers**: EventBridge Scheduler for idle/PING sweeps; per-connection TTLs.
+- **Infra**: **AWS CDK v2** (TypeScript). dynamodb-local + localstack for tests.
+
+### 6.3 Matrix: same runtime contract, different substrates
+
+| Capability         | CF adapter                          | AWS adapter                              |
+|--------------------|-------------------------------------|------------------------------------------|
+| Connection identity| DO name from connectionId           | APIGW connectionId                       |
+| Long-lived socket  | Hibernatable WebSocket in DO        | APIGW persistent WS (≤2h)                |
+| Send to client     | `ws.send` in target ConnectionDO    | `ApiGatewayManagementApi.postToConnection`|
+| Channel authority  | ChannelDO (single-threaded)         | DynamoDB `ChannelMembers` + transactions |
+| Nick uniqueness    | RegistryDO                          | DynamoDB conditional Put on `Nicks`      |
+| Cross-instance pub/sub | stub() DO→DO                   | DynamoDB query + PostToConnection fanout |
+| Timers/idle        | DO alarms                           | EventBridge Scheduler                    |
+| Durable storage    | DO transactional storage / D1       | DynamoDB                                 |
+| Secret config      | Worker secrets / KV                 | Secrets Manager / SSM                    |
+
+---
+
+## 7. Roadmap (TDD: Red → Green → Refactor per item)
+
+Each task below starts with a failing test. Phases are shippable milestones.
+
+### Phase 0 — Foundation
+1. Monorepo: pnpm workspaces + TS project references + turbo.
+2. `vitest`, coverage (c8/v8), `biome`, CI skeleton (typecheck/lint/test/coverage gate).
+3. `irc-core/protocol`: message types, numeric registry, parser golden vectors.
+4. `irc-core/protocol`: serializer with round-trip property tests.
+
+### Phase 1 — Pure protocol engine (no cloud)
+5. State shapes + `Effect` taxonomy + `dispatch` interpreter.
+6. Reducers (one file/PR each, test-first): `PING/PONG`, `QUIT`, registration
+   (`NICK`/`USER`/`CAP`/`PASS`), `JOIN`, `PART`, `PRIVMSG`/`NOTICE` (channel +
+   private), `TOPIC`, `NAMES`, `MOTD`, `MODE` (channel + user), `KICK`,
+   `INVITE`, `WHO`, `WHOIS`, `LIST`.
+7. IRCv3: `CAP LS/REQ/ACK/NAK/END`, `message-tags`, `server-time`,
+   `echo-message`, `multi-prefix`, `extended-join`, `chghost`, `away-notify`,
+   `invite-notify`, `batch`.
+8. SASL: `PLAIN` auth via pluggable `AccountStore` port; `EXTERNAL` stubbed.
+9. isupport (`005`) generator from server config.
+10. Flood control (token bucket) as a pure reducer wrapper.
+
+### Phase 2 — Runtime port + reference impl
+11. Define `IrcRuntime`; implement `in-memory-runtime`.
+12. `ConnectionActor`: bytes → framed messages → reducer → dispatch.
+13. `apps/local-cli`: a runnable WebSocket server using in-memory runtime
+    (great for manual testing and e2e fixtures).
+14. Integration test suite: scenarios expressed once, runnable against any
+    `IrcRuntime` (parametrized: in-memory, then CF, then AWS).
+
+### Phase 3 — Cloudflare adapter
+15. `ConnectionDO` (hibernatable WS), state migration, alarms for PING/idle.
+16. `RegistryDO` (with sharding plan).
+17. `ChannelDO` (roster + modes + fanout via ConnectionDO stubs).
+18. `CfRuntime` implementing `IrcRuntime`.
+19. Deploy pipeline (wrangler), staging env, smoke e2e.
+20. Docs: `deployment-cf.md`.
+
+### Phase 4 — AWS adapter
+21. CDK stack: APIGW WebSocket, Lambda, DynamoDB tables, IAM.
+22. `$connect`/`$disconnect`/`$default` handlers; `AwsRuntime`.
+23. Transactions for membership; gone-connection sweep.
+24. EventBridge idle/PING timers.
+25. Deploy pipeline (CDK), staging env, smoke e2e.
+26. Docs: `deployment-aws.md`.
+
+### Phase 5 — Cross-cutting
+27. Observability: structured logs, trace IDs, CF analytics / CloudWatch dashboards.
+28. Security: server password, SASL, hostmask cloaking, rate limits, max
+    channels/connections per user, input length caps (512-byte line per spec).
+29. Config: MOTD, server name, network name, allowed channel prefixes, max
+    clients, oper credentials.
+30. CI hardening: coverage gate ≥90%, contract tests run on every PR, mutation
+    testing (e.g., stryker) spot-check on core.
+
+### Phase 6 — Hardening / polish
+31. Load test: 10k concurrent connections per platform; capture bottlenecks.
+32. Compatibility sweep: WeeChat, HexChat, IRCCloud, TheLounge, matrix-IRC bridge.
+33. ADRs for the irreversible choices (state-authority split, effect system,
+    DO sharding, DynamoDB schema, wss-only v1).
+
+---
+
+## 8. Testing strategy
+
+| Layer            | Tooling                                    | What it asserts                       |
+|------------------|--------------------------------------------|---------------------------------------|
+| Parser/serializer| vitest + property tests (fast-check)       | Grammar correctness, round-trip       |
+| Reducers (core)  | vitest, pure unit tests                    | `(state,msg) → {state,effects}` exact |
+| Runtime contract | vitest parametrized over `IrcRuntime`      | Same scenarios pass in-memory+CF+AWS  |
+| CF adapter       | `vitest-pool-workers` (real workerd)       | DO behavior, alarms, stub fanout      |
+| AWS adapter      | vitest + dynamodb-local / localstack       | DynamoDB schema, transactions, fanout |
+| E2E              | real WS clients in `tools/irc-client`      | RFC-shaped flows vs deployed stack    |
+| Load             | `tools/load-test` synthetic client farm    | Concurrency, latency, drop rate       |
+
+**Determinism rules**: inject a `Clock` and deterministic ID/nonce ports.
+No real timers/random in reducers. Every test independent (fresh state).
+
+---
+
+## 9. Key technical decisions (proposed, to confirm)
+
+- **Monorepo**: pnpm + TypeScript project references; **turbo** for caching.
+- **Test runner**: **vitest** (ESM-native, fast, fits Workers).
+- **Lint/format**: **Biome** (single tool, fast). Fallback: ESLint + Prettier.
+- **Coverage**: v8/c8, gate ≥90% in CI; 100% on `irc-core`.
+- **CF bundler**: wrangler (esbuild internally). **AWS**: esbuild for Lambda
+  bundles, CDK for infra.
+- **Runtime**: Node 20 (Lambda) and latest Workers runtime; both ES2022+.
+- **Framing**: wss-only, one IRC message per WebSocket text frame, also accept
+  `\r\n`-joined frames (TolerantReader).
+- **Time/IDs**: injected ports (`Clock`, `IdFactory`) for deterministic tests.
+- **Secrets**: never logged; `hostmask cloaking` for user IPs.
+
+Open questions to resolve before Phase 3:
+- Channel DO sharding granularity (one DO per channel vs. bucketed)?
+  Default: one DO per channel; revisit if hot-channel limits bite.
+- DO WebSocket hibernation vs. always-on (cost vs. complexity). Default:
+  hibernation on.
+- AWS region strategy (single vs. multi-region with DynamoDB global tables).
+  Default: single region for v1.
+
+---
+
+## 10. Risks
+
+| Risk                              | Mitigation                                                |
+|-----------------------------------|-----------------------------------------------------------|
+| DO fanout latency on big channels | Batch stub calls; consider a per-DO send-list with one hop|
+| APIGW 2h connection cap (AWS)     | Reconnect protocol; client MUST tolerate `PONG`-driven reset |
+| Nick race conditions              | Single-authority Registry; conditional writes; no optimistic checks |
+| 512-byte line overflow            | Hard cap in parser; split or drop per spec                |
+| Fanout thundering herd on LIST    | Throttle + cache; cap response size                       |
+| Cross-adapter drift               | Parametrized contract tests over `IrcRuntime`             |
+| Cost runaway (DynamoDB)           | Single-table design where possible; RCU sizing in CDK     |
+
+---
+
+## 11. Deliverables checklist (definition of done for v1)
+
+- [ ] `irc-core` passes full unit + property suite at 100% coverage.
+- [ ] `irc-server` + `in-memory-runtime` pass the parametrized contract suite.
+- [ ] `apps/local-cli` connects from a real client end-to-end.
+- [ ] `cf-adapter` deployed to staging; e2e green from ≥2 real clients.
+- [ ] `aws-adapter` deployed to staging; e2e green from ≥2 real clients.
+- [ ] Compatibility sweep passed against ≥3 reference clients.
+- [ ] Load test report for each platform.
+- [ ] ADRs written for all irreversible decisions.
+- [ ] README with quickstart for both platforms.