redweb 0.12.0 → 0.13.0

package/docs/ROOM_AUTHORIZATION.md

@@ -0,0 +1,49 @@
+# Private rooms without custom socket plumbing
+
+A room is a list of subscriptions, not an identity provider. Authentication answers **who is this connection?** A room policy answers **may that identity enter this room?** Only a literal `true` grants entry.
+
+Configure `rooms.authorize(context, roomId)` on a `SocketRoute`, then use `await socket.enterRoom(roomId)` in the corresponding handler. URL paths still choose routes and message `type` still chooses handlers; there is no second action dispatcher or socket decorator system.
+
+## A complete shared-identity example
+
+The [runnable page and room example](snippets/room-access.tsx) uses one authentication function for the rendered page and `/team` socket route. It generates a fresh local-demo bearer credential on each run, has a protected page, exposes a `join` handler, and revokes page sessions and room memberships together. Its exact source is also shown on the generated examples page. The library's package gate compiles it with both standard and legacy decorators, removes its TypeScript source, and verifies real HTTP/WebSocket access, publication, and revocation against the extracted package.
+
+To run locally, save that file as `src/app.tsx` in an initialized realtime starter, then run `npm run build` and `npm start`. Send the printed `Authorization: Bearer …` header to `http://127.0.0.1:8181/` or `ws://127.0.0.1:8181/team`; the socket accepts `{"type":"join"}`. The existing starter's counter tests describe a different application, so do not treat them as acceptance tests for your modified app. Never publish the printed token or use this demonstration as a production credential service. A normal browser WebSocket cannot set an Authorization header; use your application's secure cookie/session integration for a browser product. The [dashboard recipe](../recipes/dashboard/README.md) demonstrates real cookies, persistent accounts and sign-out.
+
+## Entry, publication, and revocation
+
+- `await socket.enterRoom(id)` and `await route.rooms.enter(id, socket)` perform the bounded policy check and then commit membership synchronously. They resolve to `false` when the connection is no longer eligible or a membership limit prevents entry.
+- Existing `socket.joinRoom(id)` and `route.rooms.join(id, socket)` remain synchronous for unprotected rooms. They throw for protected rooms, with guidance to use asynchronous entry; they cannot bypass the guard. Use `enterRoom` for new guarded application code.
+- Denial, timeout, cancellation, and authorization capacity exhaustion reject entry with safe diagnostic codes. The normal handler boundary sends these to the client without disconnecting it. Broken policies remain sanitized application failures, not disguised permission denials.
+- On protected rooms, `socket.roomBroadcast` and `rooms.broadcastFrom(socket, …)` require the sender to be a current, live member. That is a membership check, **not** an application-specific write-role policy. Validate and authorize actions such as moderator announcements separately.
+- `route.rooms.broadcast` is privileged server publication. Do not expose a client-selected room through it without your own authorization. It publishes only to current eligible members and rechecks membership after serialization.
+- A grant lasts until leave, disconnect, replacement, clear, or shutdown. Changing a policy does not automatically unsubscribe existing readers. Invalidate credentials/permissions first, then call `leaveRoom`, `rooms.leave`, or `rooms.leaveAll` on affected connections. Those operations also cancel pending entry. A late policy completion cannot silently rejoin the connection.
+- `leaveAll` removes all of that connection's memberships before firing policy cancellation callbacks. `clear` removes every membership before cancellation. Nested cancellation/clear cannot reopen entry while the outer operation is still cancelling work.
+
+Redweb's `LiveHtmlServer.revoke(principal)` manages its own page lifetimes; it does not automatically revoke custom raw socket routes. The complete example explicitly invalidates its shared credential, removes the raw route's memberships, then revokes page sessions. Application storage and cross-process invalidation remain application-owned.
+
+## One request-context shape
+
+`RequestContext` and `RedWebRequest` are shared public types. Page callbacks receive `LivePageRequestContext`; enabled raw socket features expose `socket.context` as `RedWebConnectionContext`. Both provide a selected request snapshot, `principal`, and cancellation `signal`.
+
+The request snapshot is captured before raw-route admission code runs. It contains path, URL, method, headers, params, query, body and a case-insensitive `get(name)` helper—no HTTP response, transport, or framework object graph. Its data is deeply frozen and bounded to 64 KiB/16 nesting levels. Raw upgrade paths and repeated query parameters are parsed from the URL. It is not an Express request, and forwarded headers do not become trusted identities automatically.
+
+Socket identity/request/protocol references cannot be replaced; application `metadata` and resumable `session` fields remain mutable. Existing raw admission object identities remain supported and application-owned, not deeply frozen by Redweb. Page identities retain their existing primitive identity contract. Never derive a trusted identity from a client message's `principal` field or mutable application metadata.
+
+Each relevant raw connection has its own signal, cancelled on disconnect, replacement, or route draining. An operation policy receives a separate bounded signal that also cancels on leave or its deadline. Socket context remains optional when all features requiring it are disabled. In Live HTML callbacks, use the supplied callback context for the application identity: the underlying transport's internal page-session principal is not that callback identity.
+
+## Resource limits and failure meanings
+
+Protected rooms retain the existing room/member/name limits and add:
+
+| Option | Default | Meaning |
+| --- | --- | --- |
+| `authorizationTimeoutMs` | 5000 | Maximum time to wait for one policy. |
+| `maxPendingAuthorizations` | 128 | Underlying policy work across the registry. |
+| `maxPendingPerConnection` | 4 | Underlying policy work for one connection. |
+
+These options require `authorize`. Concurrent pending requests for the same connection/room share one check. No room or membership is reserved while permission is pending; final insertion rechecks all capacities. Timed-out or cancelled policy work stays charged until the **actual application promise settles**, so an uncooperative policy cannot spawn unlimited background work. Honor the signal and use bounded downstream I/O; policies that never settle can exhaust capacity until corrected/restarted. JavaScript's synchronous execution cannot be preempted.
+
+`ACCESS_DENIED`, `ACCESS_TIMEOUT`, `ACCESS_CANCELLED`, and `ACCESS_CAPACITY` indicate no membership was committed by that failed entry. They do not assert that an external policy had no side effects. Protocol routes return the standard error envelope and request ID; unversioned routes return `{ code, error }`. A policy exception becomes sanitized `HANDLER_FAILED` and follows the normal application-error close behavior. Nothing automatically retries entry or promises exactly-once application delivery.
+
+Use HTTPS/WSS, trusted origins for browser credentials, real session expiry, input limits, and persistent application authorization before public deployment. Process-local rooms and grants do not become distributed merely because a socket route has a distribution adapter.

package/docs/RUNTIME_DIAGNOSTICS.md

@@ -0,0 +1,78 @@
+# Understand failures before retrying
+
+Status: unreleased on `codex/agent-ready`; these handshake diagnostics are not in npm 0.12.0.
+
+Authentication identifies a visitor. Authorization decides what that visitor may do. Validation checks an input's shape. An application failure means server code or a dependency failed; it is not evidence that the visitor supplied bad credentials.
+
+Redweb keeps these boundaries separate. It does not automatically retry failed actions, undo application writes, or guarantee exactly-once delivery.
+
+## Reading a failed connection
+
+Before a WebSocket opens, Redweb sends an HTTP rejection with a fixed `Redweb-Error` header and `Cache-Control: no-store`. There is no error body and no callback exception text. Node's `ws` client can inspect it:
+
+```typescript
+import WebSocket from 'ws';
+
+const socket = new WebSocket('ws://127.0.0.1:8181/match?redwebVersion=1');
+socket.on('unexpected-response', (_request, response) => {
+    console.error(response.statusCode, response.headers['redweb-error']);
+    response.resume();
+    socket.terminate();
+});
+socket.on('error', () => console.error('Connection did not open.'));
+```
+
+This is a Node diagnostic example, not browser code. Native browser WebSocket JavaScript cannot inspect handshake status or response headers. Use the browser network inspector during development and your application's normal HTTP sign-in/status flow for user-facing guidance. A generic browser socket error alone cannot distinguish rejected credentials from networking, origin, protocol, or server failures. Proxies may replace or strip responses.
+
+| Code | HTTP status | Meaning and next step |
+| --- | --- | --- |
+| `REQUEST_INVALID` | 400 | The upgrade request cannot be represented safely. Correct the request or server middleware. |
+| `AUTHENTICATION_REQUIRED` | 401 | Identity was rejected, or the page session is missing, expired, already attached, or mismatched. Obtain valid credentials/a fresh page; do not retry the same rejected credentials in a loop. |
+| `ORIGIN_DENIED` | 403 | Browser origin was missing or not allowed. Correct the trusted-origin configuration; do not disable origin checks to hide the failure. |
+| `ACCESS_DENIED` | 403 | The page permission policy denied access. Obtain permission before retrying. |
+| `PLACEMENT_DENIED` | 403 | Placement explicitly rejected the connection. Follow application placement rules. |
+| `PROTOCOL_UNSUPPORTED` | 426 | Negotiation requires a supported version. `Redweb-Versions` lists supported versions; use a compatible client and contract. |
+| `AUTHENTICATION_FAILED` | 500 | The rendered-page identity callback failed. Investigate the application or identity provider. |
+| `ADMISSION_FAILED` | 500 | Admission/origin/placement code, a page upgrade policy, or the upgrade pipeline failed. This is not a bad-password response. |
+| `PLACEMENT_INVALID` | 500 | Placement returned an unsafe or disallowed redirect. Repair the server-side placement result/allowlist. |
+| `AUTHENTICATION_TIMEOUT`, `ACCESS_TIMEOUT`, `ADMISSION_TIMEOUT` | 503 | The relevant stage exceeded its deadline. Check the dependency and use bounded reconnect backoff only when appropriate. |
+| `AUTHENTICATION_CANCELLED`, `ACCESS_CANCELLED`, `ADMISSION_CANCELLED` | 503 | The relevant lifetime ended. Start a new permitted attempt rather than reusing a revoked page/session. A disconnected peer may receive no response. |
+| `ACCESS_CAPACITY`, `ADMISSION_CAPACITY` | 503 | Bounded authorization/admission work or connection capacity is exhausted. Wait and back off; do not retry in a tight loop. |
+| `SERVER_DRAINING`, `ROUTE_UNAVAILABLE` | 503 | The service is draining or the route is not ready. Reconnect to a ready instance according to the application's routing policy. |
+
+Accepted placement redirects remain HTTP 307 with the validated `Location`, no error code, and no-store caching. A redirect is not proof that the destination will admit the same credentials. Never forward credentials to arbitrary redirect destinations.
+
+Raw route authentication preserves its existing contract: only literal `false` rejects the identity; application-owned principal objects remain supported. Page authentication requires its documented primitive identity. Do not rely on a raw callback returning `undefined` to deny access.
+
+## Page requests
+
+Page HTTP failures return `{ "error": { "code": "...", "message": "..." } }` with `Cache-Control: private, no-store`. Authentication and authorization use the categories above. `PAGE_CAPACITY` is 503; `PAGE_FAILED` is a sanitized 500 for construction, loading, or rendering failures, including public pages. Unknown application error text and Express development stacks are not returned. Typed errors are reconstructed from the fixed catalogue rather than trusting mutable status/message fields.
+
+If the response is already closed, Redweb does not write another response. If headers were already sent by application middleware, the connection is closed instead of appending a misleading JSON error. Redweb cannot retract content your middleware already sent or sanitize arbitrary HTTP routes you mount yourself.
+
+## Actions and established sockets
+
+Once connected, failures use the existing protocol error envelope and request ID when available. Unversioned room-entry failures use `{ code, error }`; other legacy unversioned failures retain `{ error }` without a structured code. The client may still disconnect before receiving the response. Typed permission/input errors are normalized again at the final send boundary, so application catch/rethrow code cannot accidentally expose appended private exception text.
+
+| Boundary | Diagnostic | What Redweb guarantees |
+| --- | --- | --- |
+| Action input | `ACTION_INVALID_INPUT` | The action method was not invoked. Correct the form values. |
+| Action input lifetime | `ACTION_VALIDATION_TIMEOUT`, `ACTION_CANCELLED` | Validation did not complete within its lifetime; the action method was not invoked. |
+| Action/room permission | `ACCESS_DENIED`, `ACCESS_TIMEOUT`, `ACCESS_CANCELLED`, `ACCESS_CAPACITY` | That failed permission check did not commit room entry or invoke the guarded action. Existing memberships and prior actions are separate. These responses do not inherently close the socket. |
+| Browser send | `ACTION_OFFLINE`, `ACTION_CAPACITY` | This browser action was not sent. Reconnect or wait before deliberately trying again. |
+| Socket envelope/contract | `INVALID_MESSAGE`, `INVALID_PAYLOAD`, `UNKNOWN_HANDLER` | The requested handler callback was not invoked. Correct the message/contract. These paths generally close the connection; they are not automatic retry signals. |
+| Application/validator/output bug | `HANDLER_FAILED` | The operation failed; application effects may already have happened. Inspect authoritative state before resubmitting. |
+
+An input validator, identity lookup, or permission callback can itself perform external work. A “method was not invoked” result does not promise those callbacks had no side effects. Keep validators and policies free of writes where practical; use explicit idempotency keys and durable transactions for application operations that may be retried.
+
+Those non-invocation guarantees describe failures produced by Redweb's validation and permission gates. They do not apply to application code deliberately throwing an internal typed error or sending the same diagnostic after its own work has begun.
+
+## Deadlines, cancellation, and disclosure limits
+
+Raw admission shares the bounded-operation implementation used by other policy/validation paths. It checks the deadline between origin, identity, and placement stages, so a timed-out or cancelled stage cannot start the next stage after eventually returning. It retains its actual evaluation promise in admission accounting until that evaluation settles. Synchronous JavaScript cannot be interrupted; a callback that blocks the event loop delays timeout observation, but an overdue result cannot admit a connection.
+
+Page identity and permission evaluation also have their own deadlines and session/revocation signals. The outer raw-admission deadline does not forcibly stop those nested callbacks or their external I/O. Cancellation of observation is not cancellation of database/network side effects. Honor available signals, set downstream timeouts, and never treat this as a sandbox for untrusted callback code.
+
+Default handler responses are sanitized. Raw routes deliberately configured with `exposeErrors: true` opt into disclosing handler exception text; do not enable that in production. Existing application/logger hooks may receive original errors and client metadata, so logs require access controls and redaction. The new upgrade pipeline logs only fixed admission failure details, and a throwing logger cannot prevent upgrade rejection or reservation cleanup.
+
+See [private rooms](ROOM_AUTHORIZATION.md), [socket contracts](SOCKET_CONTRACTS.md), and [operating socket services](MULTIPLAYER_OPERATIONS.md) for their complete limits. A successful `send` means accepted by the local transport, not acknowledged application delivery.

package/docs/SERVER_RECOVERY_CANDIDATE.md

@@ -0,0 +1,185 @@
+# Server-focused recovery candidate
+
+Status: **adopted as the blocking CI recovery check** after independent review,
+five clean cross-runtime results and explicit maintainer authorization to continue.
+Original `npm run verify:recovery` keeps its command and runs in CI as an explicitly
+non-blocking diagnostic. A later [exact-byte boundary correction](ORIGINAL_RECOVERY_VERIFICATION.md)
+fixes false rejection at precisely 110%; its workload and limit are unchanged.
+Historical failures are not erased by a server pass or that rounding correction.
+
+## Contract: server-steady-v1
+
+The candidate reuses the existing split coordinator and workers in baseline mode,
+with separate coordinator, Redweb server and native `ws` load-generator processes.
+It preconditions with 1,200 connections, warms with 200, then executes five storms
+of 1,200 each, in batches of 50: 7,400 total exact request/reply exchanges.
+Each batch reconciles client sends/replies and server receives and waits for
+server connection cleanup. Rooms and sessions are checked at phase boundaries,
+not claimed empty after every batch. Sampling settles for 400 ms and performs the
+existing two explicit garbage collections before reading retained heap.
+
+Every storm's server heap must be at most 110% of the **same** warmed server heap.
+A failed middle storm fails even if the final storm recovers. Client heap ratios
+are reported separately without a client heap acceptance budget. Exact delivery,
+empty registries, stable runtime/process identities, normal worker exits, closed
+output pipes, complete logs and unchanged input hashes remain mandatory for both
+roles. A forced cleanup cannot produce a passing candidate.
+
+This is a different measurement from the shared-process gate, not proof that its
+failure was harmless. No code-memory subtraction, moving baseline, retries until
+green, instrumentation, snapshots or tuned workload overrides are permitted.
+The reused worker reads V8 statistics after its heap sample; that existing
+observation and split-process scheduling are part of this candidate protocol.
+
+Reports retain the reviewed `candidateOnly: true` / `candidatePassed` field names
+for format compatibility with the collected evidence. CI now adopts the command's
+exit status as this scoped recovery gate; these legacy field names do not mean
+whole-release approval. No workload, threshold, worker or report-schema change
+was needed for adoption. The original measurement keeps its own exit status;
+CI records the raw step outcome, emits a warning on non-success, and retains logs
+alongside server samples, process exits and input/output hashes for 30 days.
+Both commands have two-minute CI deadlines and run sequentially, without retries.
+The diagnostic only starts after a passing server gate confirms worker cleanup;
+after a failed/timed-out server run it is explicitly skipped, not called passing.
+
+## Run the candidate
+
+From the matching Redweb source checkout (not an installed application), run
+`npm run verify:recovery:server` in a
+clean environment. It creates an exclusive directory under `coverage/` and
+prints its location. An optional absolute, nonexistent directory can be supplied
+after `--`. Existing evidence is never overwritten.
+
+The command rejects Node flags, nonempty `NODE_OPTIONS`, `NODE_V8_COVERAGE` and
+`REDWEB_RECOVERY_*` overrides (including differently cased environment names).
+Reports include source hashes, all phase samples, observed worker exits and log
+digests. Failures retain partial evidence; primary errors are not replaced by
+secondary finalization errors. A heap-budget failure exits nonzero just like an
+invalid-evidence failure, but is identified separately in its saved report.
+
+Policy units use explicitly synthetic reports for malformed evidence and exact
+budget boundaries. Real integration uses a managed coordinator process and real
+workers/sockets; its parent bounds the coordinator and separately reaps registered
+workers, whose POSIX process groups are independent. Missing registration or
+uncertain cleanup fails and retains the workspace rather than claiming complete
+tree cleanup. Coordinator-only
+instrumentation is used for behavioral coverage, never claimed as a clean
+candidate measurement. Shared worker lifecycle tests remain separate. No mocked
+transport, garbage collector, process or filesystem replaces these integration
+checks.
+
+## Maintained acceptance-tool coverage
+
+The current gate also includes the full authored split coordinator/worker:
+161 tests, all-four 100% over five source files. See
+[authored coverage](SPLIT_RECOVERY_COVERAGE.md) for current maps, boundaries and
+the separate full-regression/hosted outcomes. The original three-file checkpoint
+below is historical and does not describe the current command's complete scope.
+
+### Original policy/coordinator/CLI checkpoint
+
+`npm run verify:recovery:coverage` now runs the existing coordinator/policy units,
+real worker integrations and three added CLI exit-boundary units together. It
+enforces all four 100% thresholds over `ServerRecoveryPolicy.js`,
+`ServerRecoveryCandidate.js` and `verify-server-recovery.js`, and is required in
+the lifecycle CI job. The command passed 76 tests across five suites in 28.656
+seconds on Windows/Node 22.21.0. The CLI units explicitly substitute the coordinator
+result and stderr to cover pass, budget-failure and rejected-promise exits; they
+are not described as mock-free integration. Actual CLI/worker/socket behavior is
+also exercised by integration tests and the ordinary CI acceptance invocation.
+This closes the earlier thin-CLI coverage gap without claiming coverage of the
+shared diagnostic workers or every private verification tool.
+The report contains 132 statements, 25 branches, 25 functions and 120 lines,
+all covered. `coverage/server-recovery-acceptance/coverage-final.json` SHA-256:
+`0d6ebe45fe2e1f87f2ddcb6afb4fe1960473c40a09c877f4697cf90db2a1ff3c`.
+
+## Implementation checkpoint
+
+The senior critic approved measurement after environment-case and test-process
+ownership findings were fixed. The scoped regression passed 73 tests in four
+suites, including existing split-worker regressions. New policy/coordinator
+coverage is all-four 100%: 128 statements, 23 branches, 23 functions and 116 lines.
+The report is `coverage/server-recovery-candidate/coverage-final.json`, SHA-256
+`17906b03fae19ed3bef54d8d82743ad831a6124fa1bd88147cbb4e2da64f87d7`.
+This does not claim whole-repository, test-helper or thin-CLI-wrapper coverage.
+Generated-content freshness and all TypeScript pretest configurations also pass.
+
+Coverage uses original coordinator source in a managed Jest child and the parent
+unit tests. It checks identical source mappings before merging only the parent's
+selected files. Child collection is non-gating; the combined scoped gate retains
+all four 100% thresholds. This is instrumentation for behavioral verification,
+not a clean candidate heap measurement.
+
+The temporary `server-recovery-candidate.yml` workflow collects one clean result
+each on Ubuntu 24.04 with Node 18.20.8, 20.20.2, 22.23.2 and 24.19.0. Its trigger
+is restricted to changes to that workflow on the implementation branch; repeated
+attempts are rejected. Each isolated job installs the locked dependencies before
+measurement, runs no competing tests, and uploads candidate evidence on failure
+as well as success. Remove the workflow after collection. This new candidate
+protocol is not a rerun of the historical original/split comparison and does not
+modify the ordinary acceptance workflow.
+
+## Collected candidate results at 15c5a4e
+
+The [single Ubuntu matrix](https://github.com/lakam99/redweb/actions/runs/33352534392)
+completed successfully on all four pinned runtimes. A separate single Windows
+Node 22.21.0 run also passed. These were clean candidate invocations, not the
+instrumented behavioral runs described above. Each verified all 7,400 exact
+replies, phase registries, unchanged inputs, complete logs and normal worker
+exits. Downloaded sample streams, worker inventories and every log's bytes/hash
+were independently reconciled with the saved reports.
+
+| Environment | Server peak % of warm | Server final % of warm | Client peak % of warm |
+| --- | ---: | ---: | ---: |
+| Ubuntu / Node 18.20.8 | 102.738879 | 101.936032 | 107.220473 |
+| Ubuntu / Node 20.20.2 | 106.724670 | 104.968229 | 111.411540 |
+| Ubuntu / Node 22.23.2 | 108.491002 | 97.235712 | 112.873352 |
+| Ubuntu / Node 24.19.0 | 100.817681 | 100.817681 | 101.139821 |
+| Windows / Node 22.21.0 | 109.253735 | 95.525155 | 113.270161 |
+
+Only the server column has the candidate's 110% budget. The client measurements
+above 110% are explicit, not subtracted or represented as passing that budget.
+One run per environment demonstrates these executions, not repeatability or
+production capacity. The original shared-process CI failures remain unchanged.
+
+Ubuntu artifacts are retained locally under
+`coverage/server-recovery-matrix-33352534392/`; hosted retention is 30 days.
+Each artifact is named `server-recovery-<node>-15c5a4eef60a7a544745a9382bf9f517c02a30f8`.
+SHA-256 for each `report.json`:
+
+- Node 18.20.8: `9899ea2201724957fc1aba88d0eb901cea0a0e7e17f43ebc3e21a7593cdca9c4`.
+- Node 20.20.2: `8b0949c154e905122091ba6a0e37019d404c1a21266406ce00b23b6c73464860`.
+- Node 22.23.2: `41f55fb4fbb1b90079d624f4602a3a4d5441391dd112c992fabdef5b720f523b`.
+- Node 24.19.0: `e7ec1c25a3c94aa6b48d30382cf044f8425fab01a09dd548fd0285e06544733e`.
+- Windows `coverage/server-recovery-local-15c5a4e/report.json`:
+  `26e90708d03537f16a1b73b5ba16c57cbee9743e4fff2c770933118ffdcf032d`.
+
+The temporary collection workflow was removed after completion; no measurement
+rerun occurred. Adoption was pending at this historical checkpoint and is now
+authorized as described above.
+
+The senior critic independently verified the actual PR commit, all five reports,
+the Ubuntu manifests against 104 committed inputs, delivery/sample/inventory
+reconciliation and log hashes. The reviewer recommends presenting adoption for
+the maintainer's decision without another experiment. The justification is
+isolating server retention from the load generator, not proving the original
+failures harmless or claiming their cause was fixed.
+
+## Full regression and unchanged CI at 15c5a4e
+
+Windows `npm test -- --runInBand --silent` completed successfully: 918 tests in
+86 suites, 447.46 seconds, including the normal generated-content/type pretest.
+The configured library scope remains all-four 100%: 5,445 statements, 4,044
+branches, 978 functions and 4,464 lines. The coverage report
+`coverage/coverage-final.json` has SHA-256
+`961dd4e6bc2e90df001df71281622140f153024ae71e4295e7b2ed92d49cb045`.
+The candidate tooling coverage is the separate scoped result above.
+
+Ordinary [push CI](https://github.com/lakam99/redweb/actions/runs/33352534344)
+passed. Ordinary [PR CI](https://github.com/lakam99/redweb/actions/runs/33352536964)
+failed only the unchanged Node 22 shared-process recovery check: storm 4 reached
+110.530483% of warm against the 110% limit; final heap was 97.045378%, with empty
+registries. Its tests and load check passed; its subsequent audit was skipped.
+Node 18/20/24 and the lifecycle/browser/package job passed. No retry occurred.
+These results preserve the distinction between a passing candidate and the
+still-failing existing acceptance contract; this is not merge approval.

package/docs/SOAK_ROTATION_OBSERVATION.md

@@ -0,0 +1,160 @@
+# Soak rotation: retained failure and controlled observation
+
+## Actual CI failure, still unresolved
+
+At `df58f94`, PR run 33432429300, Node 24 job 99620736610, failed the native
+ten-second mechanics test's delivery assertion: **98.47715736040608%**, below
+the unchanged 99% limit. The job recorded 1,939 passes, one failure, eight skips,
+175 suites and 1,212.024 seconds, despite all-four 100% library coverage.
+The matching push and later Windows run passed; they do not erase this failure.
+
+The previous test saved its measurement only after all assertions. Its temporary
+workspace was then deleted on failure, so the original raw JSON and exact
+sent/received counts are unavailable. Do not infer those counts from the ratio.
+The complete failed-job log was retained with ANSI removed and LF newlines at
+`coverage/ci-df58f94-node24-99620736610.log`, SHA-256
+`170eec5766098b6d9b27cfbebaf56ace15984d964fe5e2327ddb7b6daea4df8a`.
+
+## Evidence preservation correction
+
+The mechanics fixture now saves raw report text, command output, observed normal
+exit status and available errors before parsing or policy assertions. Invalid
+JSON and a sub-limit outcome remain inspectable. Unknown launch/timeout/cleanup
+failures remain failures, not fabricated exit statuses. If reading or writing
+evidence fails, the original workspace is retained and primary errors survive.
+The critic caught the outer-workspace deletion case; real-filesystem regressions
+now verify the original raw file survives after `VerificationWorkspace.run()`
+rejects. Test cleanup explicitly removes its own retained fixtures afterwards.
+
+The GitHub Node matrix now uploads available `coverage/soak-tools/smoke-reports/`
+artifacts on success or failure, independently of the lifecycle job's coverage
+artifact. Missing evidence warns because a pretest/launch failure can occur
+before the measurement exists; that warning does not turn failed tests green.
+Retention is 30 days. Each observation gets a new exclusive filename.
+
+## Controlled real-socket comparison
+
+Two native WebSocket cases hold one actual reply after the peer receives its
+tick. They use no mocked transport, timers or process APIs:
+
+- Reply before rotation: the original reply arrives, rotation closes the old
+  socket and opens its replacement, and both later replies arrive: **4 sent,
+  4 received**.
+- Rotation before releasing the held reply: the old socket is closed before
+  replacement; releasing that reply fails on the closed peer. Both later
+  replies arrive: **4 sent, 3 received**, with exactly one still missing.
+
+Assertions verify the pending tick before rotation, closed original socket,
+event order, distinct replacement, generation increment and exact counters.
+The targeted comparison passed both cases in 0.582 seconds.
+
+This proves a possible loss mechanism during intentional rotation, **not the
+cause of the historical CI run**. The existing 100 ms send and 1,000 ms rotation
+timers can overlap with pending work. No drain, timing, workload or delivery
+limit has been changed, and missing replies remain in the denominator. The
+next ordinary CI observation must be retained before making a stronger claim.
+A future drain policy would change rotation semantics and needs its own tested,
+explicitly documented methodology; it is not a silent fix for this result.
+
+See [the original soak-verifier correction](SOAK_VERIFICATION.md) for policy
+details and the distinction between short mechanics tests and hour acceptance.
+
+## Final verification of this correction
+
+The maintained `npm run verify:soak:coverage` passed 91 tests across five suites
+in 16.992 seconds after the retention review fix. All 280 statements / 116 branch
+outcomes / 68 functions / 200 lines are covered across the three unchanged
+soak modules and the new test-only retention helper. The latter contributes
+39 statements / 16 branches / three functions / 27 lines; it is not shipped
+runtime coverage. Map: `coverage/soak-tools/coverage-final.json`, SHA-256
+`19d9f505a2d2156e09177a86e5a922fd848b5e1fcdb14c0033f6fab889a98377`.
+
+The reviewed nine-test retention selection also passed independently in 2.266
+seconds with all-four 100% helper coverage. It includes actual child exits and
+filesystem failures, plus explicit application callback faults.
+
+A follow-up bounds each of the three real child-command cases at ten seconds,
+with a 40-second outer test budget allowing independent process/pipe/workspace
+cleanup to finish first. The unchanged nine cases passed again in 0.745 seconds
+at all-four 100% helper coverage; no soak workload was rerun for this test-budget
+correction.
+
+Two ordinary mechanics observations were retained during implementation:
+
+- Before the outer-retention fix: 163 sent / 162 received, 99.38650306748467%,
+  command exit 1 for the known room-phase trend (early 1, late 2), not delivery.
+  Observation `2bed814f-ca66-438e-b3a4-1ab338848976.json`, SHA-256
+  `6b2ff5305c42c103a878a304aaadd353d9225710539674a749d8fa636a4c2af8`.
+- Final wiring verification: 173 sent / 172 received, 99.42196531791907%,
+  command exit 0. Observation `8aacb5b0-9512-48ac-a201-7b64255af574.json`, SHA-256
+  `bee152a47e3047ce9d9bd20570d1b3e253359a315e7cc259847c0acd76421760`.
+
+Both are under `coverage/soak-tools/smoke-reports/`. The prior fixture's explicit
+room-phase allowance is unchanged: a mechanics-test pass can retain command
+exit 1 and is not soak acceptance. Neither observation is lossless, neither
+supersedes the hosted sub-99% failure, and neither is a new hour soak.
+
+## Current-head long-run failure and heartbeat correction
+
+At `1bb61c3`, a fresh default hour began at 21:07:04Z under a one-shot Windows
+Scheduled Task with independently reviewed child ownership. The workload exited1
+after about nine minutes with `Soak client disconnected unexpectedly.` No report
+was created, so it provides no partial delivery, resource-trend or acceptance
+measurement. The owner exited normally, did not force termination, retained its
+1,520-byte stderr and terminal outcome, and did not retry. The exact task was
+removed only after terminal validation. Outcome SHA-256:
+`53083685f019d82d5f62421faaa80dd15f16d06a3d85ff6b3b08e3546982e258`;
+stderr SHA-256:
+`951a06f05779be74031fb2f7736645557593b1cc6f1b750736b77a8348241131`.
+
+The old soak client discarded the native close code and reason. Unexpected close
+events now preserve both, with reasons escaped for unambiguous line-oriented logs;
+unit and real-WebSocket checks cover framework policy and transport closes,
+including absent codes, without changing delivery accounting. An earlier error
+remains the primary failure if its event arrives before close. The maintained
+soak gate passes94 tests at all-four100% across its existing four-file scope.
+
+A separate no-mock regression deterministically reproduced one possible false
+disconnect: a responsive same-process client automatically pongs, while one
+server-side ping callback stalls the event loop beyond the heartbeat deadline.
+Previously the next timer terminated that healthy client before its already
+dispatched pong handling could win. Heartbeat expiry now owns one deduplicated,
+unreferenced `Immediate` per expired socket. The deferred check terminates a peer
+that is still silent; pong handling, detach/reattach, or monitor shutdown makes
+the stale check harmless. No deferred-check allocation occurs on healthy ticks and no timeout is
+reset. Connection/queue limits remain the resource bounds.
+
+The focused heartbeat scope passes71 unit and real-socket tests at100% statements,
+branches, functions and lines. The senior critic required and approved silent-peer,
+deduplication, detach/reattach and shutdown ownership cases. This establishes a
+real possible mechanism and its correction, **not the cause of the failed hour**:
+that run's missing close code cannot be recovered.
+
+## Corrected-hour outcome
+
+An immutable `11d1dcc` candidate then completed a fresh default hour from
+2026-08-31T21:39:31.146Z to 22:39:35.639Z on Windows and Node22.21.0. A one-shot,
+no-trigger Scheduled Task owned the hidden process. No competing local verification
+workload ran. The task finished ready with result0, the child exited0 without
+forced termination or cleanup error, stderr was empty, and the exact task was
+removed only after independent validation.
+
+- 3,600 seconds,64 clients and720 samples.
+- 2,106,821 messages sent;2,106,745 received;76 missing:99.9963926693%
+  delivery against the unchanged99% requirement. This is not lossless delivery.
+- Warm heap11,570,488 bytes; peak12,129,384; final11,293,216:97.6036274356%
+  of warm against the unchanged110% limit.
+- All eight resource trends passed; final client, room, session and in-flight
+  registries were empty. Handles moved0→1, within the allowed one.
+- The report and stdout match byte-for-byte at SHA-256
+  `0ab7b67adfa23469cbccd546782102650206a2052c2952b3bf21556cecb802b8`;
+  stderr contains zero bytes.
+- All112 raw runtime/dependency/soak inputs matched their recorded hashes after
+  execution. An independent recomputation also matched the582-file tracked-tree
+  digest `39d0b28ae6554467f6b7737c1ada246a51e1cf85dc672e8406796b7ca86983b3`.
+  The launcher itself did not recompute that aggregate digest.
+
+Report: `coverage/soak-11d1dcc-scheduled-20260831.json`. The subsequent
+`136bbaf` review increment strengthens proof and diagnostics without changing the
+production heartbeat implementation. Because it changes one soak-owned diagnostic
+input, this hour remains evidence for `11d1dcc`, not a final-head soak claim.

package/docs/SOAK_VERIFICATION.md

@@ -0,0 +1,154 @@
+# Soak-verifier correction
+
+This increment changes verification tooling, not Redweb's production runtime.
+It closes a private-tool coverage gap without waiving the separate throughput
+benchmark or claiming a new one-hour soak.
+
+## Reproduced weakness and correction
+
+The original command ran against real sockets for ten seconds with two clients
+and a 20-second sample interval. It exited successfully with only two samples.
+Its final heap became its own warm baseline (exactly 100% reported), and all
+eight trends compared the initial sample with itself. The original script SHA-256
+was `ce30f8abcda0e018b8c586bd53dba775aebdaaaaf5275aa8bdf2b2f8d9b1d22b`.
+The retained report `coverage/soak-original-sparse.json` has SHA-256
+`fc8cf5d93237e27f91f7c3ac8838682ac4b8ec629d1c0ed65ef3652ea8ad4eaa`.
+
+The policy now requires safe native timer ranges, array/derived exchange-count
+capacities and enough actual active-phase samples. The socket owner records every
+transport immediately, settles parallel acquisition, and matches the existing
+`{tick}` replies against each connection's pending ticks exactly once. Arbitrary
+frames, duplicate/unsent ticks and unexpected disconnects fail instead of inflating
+delivery or silently reducing the workload. Intentional rotation/cleanup remains
+distinct, and lost replies remain counted as missing.
+
+Traffic/sample failures and rotation errors signal the coordinator immediately.
+Timers stop, pending rotation settles, drain/client closure/expiry sampling/server
+shutdown are attempted independently, and primary plus cleanup failures survive.
+Output follows cleanup; an existing output file cannot be overwritten, and a
+failed file write cannot be preceded by misleading success output.
+
+## Preserved measurement boundaries
+
+- Defaults: 3,600 seconds, 64 clients, five-second samples; 100 ms traffic and
+  one-second rotation cadence; original route limits and unchanged wire payloads.
+- Original GC sampling sequence, upper medians, trend windows and eight growth
+  budgets; at least two active-phase periodic samples are now required.
+- Final heap is sampled after client closure/expiry but before server shutdown.
+  The later two GC calls do not update the reported final heap.
+- The existing 99% delivery, 110% final heap and baseline-plus-one handle limits
+  remain. Integer count/byte comparisons avoid floating-point boundary errors;
+  the displayed ratios are retained. A genuine 1,000→1,100-byte fixture previously
+  calculated `110.00000000000001` and falsely failed the exact 110% boundary.
+
+Explicit pending sets/listeners/ownership change the harness's allocations and
+parsing work. These results are not byte-identical historical measurements or a
+claimed runtime optimization. Short integration workloads establish mechanics,
+not production capacity or long-run stability.
+
+## Verification
+
+Windows / Node 22.21.0: `npm run verify:soak:coverage` passes 80 tests/four suites
+in 14.494 seconds, requiring all-four 100% across three files: 241 statements,
+100 branches, 65 functions and 173 lines. Native WebSocket/process tests cover
+valid traffic, rotation, malformed/duplicate/unsent replies, server-error frames,
+unexpected closes, partial acquisition, failed replacement, missing GC, sparse
+sampling and actual short-run reports. Explicit boundary units cover policy,
+timers, cleanup faults, exact limit acceptance and output suppression. No API mocks
+are used in the native integration tests.
+
+The first client-unit draft used cross-realm `instanceof Error` against Node's
+AssertionError. It was corrected to Node's native-error check; no production
+change was made for that fixture issue. The critic's exact-boundary, derived
+counter capacity and supervision findings were corrected. The native rotation
+test allows 80 seconds for all operation/cleanup phases. CI allows 20 minutes for
+the complete scoped suite and retains reports for 30 days on failure or success;
+the separate 30-second/16-client CI soak has two-minute external supervision.
+
+| Source | SHA-256 |
+| --- | --- |
+| `scripts/verify-soak.js` | `1a2b1a7bebc81b0b95e078774bc0c9b534d70b58f852caad1b564f508151c91c` |
+| `scripts/lib/SoakClients.js` | `b00458a3dd1970124107e3c6990848af738f080e6ea3f2c7b56e9e952a4c22a5` |
+| `scripts/lib/SoakMeasurement.js` | `0883d6d5ef6c92b8a2161274b1dbe86187795a629de0705f6e6928f812d7424a` |
+
+Report `coverage/soak-tools/coverage-final.json` SHA-256:
+`a32bc0a90a69aba60c727e060df20e5fe90181335b618e96a963cb69d028a4c8`.
+
+An initial clean 30-second/16-client run before the review follow-ups passed:
+4,365 sent, 4,364 received, one missing (99.97709049%), seven samples, all eight
+trends stable, zero registries, final heap 100.09062623% and handles 1→2.
+Report `coverage/soak-corrected-20260831-0346.json` SHA-256:
+`860351a6038c9a43c5a0190ff98400c88cdf3de0201a20eaacfda721d8b8a73f`.
+
+After final scoped tests exited, the corrected 30-second/16-client run passed:
+4,368 sent, 4,365 received, **three missing** (99.93131868%), seven samples, all
+eight trends stable, zero registries, final heap 99.78205308% and handles 1→2.
+Raw evidence `coverage/soak-corrected-final-20260831.json` has SHA-256
+`9816802b52c68a774b232e23871fed16609baa88cd9c9a7c7e4809ac357f0113`. Neither run is
+lossless. Hosted checks for this increment remain pending.
+No npm publication, deployment, merge or new 60-minute soak occurred.
+
+The full regression selected at `31fa9b2` passed 1,246 tests/119 suites in
+653.569 seconds, with two POSIX-only skips on Windows. All 91 library files retain
+100% coverage: 5,449 statements, 4,046 branches, 978 functions and 4,468 lines.
+Library and soak-verifier sources remained unchanged throughout. Five later
+application-recorder units passed separately and are not included in 1,246.
+Generated-content/type checks pass, and the critic approved all 15 actual remote
+blobs on PR16. Hosted PR33370390186 and push33370386741 were still running when
+this evidence was recorded; completed lifecycle/package jobs are not a complete
+workflow pass.
+
+Full map `coverage/coverage-final.json` SHA-256:
+`78ffc666b8181154cf5a6c2b9673e82b6f65b50d6c5fbb29d02941c96d598d0e`.
+Inventory `coverage/soak-full-results.json` SHA-256:
+`2ac4d0bc14ec7fee441a38f710ac1234b68ceb7c94d1280523df3acaa9b2304e`.
+
+## Subsequent hosted room-phase failure
+
+Both workflows at `31fa9b2` eventually completed successfully. At `d29be8c`,
+push run 33371370687 passed, but [PR run 33371374433](https://github.com/lakam99/redweb/actions/runs/33371374433)
+failed its Node 20 ten-second/two-client mechanics fixture. That failed result
+is not replaced by the passing sibling: 198 sent/198 received, no missing replies,
+11 samples, zero final registries and final heap 103.71149824% of warm. The sole
+failed trend was rooms: early 1, late 2, delta 1, peak 2, allowance 0, monotonically
+increasing. The saved summary lacks phase samples, so the exact CI timing cannot
+be reconstructed.
+
+A new deterministic native test uses real Redweb room handling and `SoakClients`:
+two clients join separate rooms; rotating one removes its sole-member room; the
+replacement connects while the room count remains one; its next tick recreates
+the room. All four replies are exact and final clients/rooms are zero. This proves
+the observable 2→1→2 phase exists, not that the historical CI run followed that
+precise sequence. Its first local fixture omitted room configuration and failed;
+after supplying the actual soak's room settings, the deterministic test passed.
+
+The ten-second test now checks measurement mechanics and the unchanged policy's
+actual exit code. It permits only that exact 1→2 room-trend failure; all other
+trends, delivery, final heap, handles and empty-registry assertions remain strict.
+Launch, timeout, malformed output, stderr and cleanup failures still fail the test.
+Its report and actual exit code are retained in
+`coverage/soak-tools/smoke-reports/` as **mechanics-only**, including in CI artifacts.
+A passing mechanics test does not relabel its nonzero measurement as acceptance.
+
+No runtime, traffic, rotation, sampling, trend budget or policy source changed.
+The separate blocking CI soak remains 30 seconds / 16 clients, and the default
+one-hour workload and all acceptance thresholds are unchanged. Neither the
+historical short failure nor the open throughput benchmark is waived.
+
+The corrected mechanics fixture passed within the maintained 80-test/four-suite
+soak coverage run (14.414 seconds), retaining all-four 100% of the unchanged three
+soak modules. The separate deterministic room-phase test passed in 0.751 seconds.
+The senior critic approved both the bounded reproduction and narrowly preserved
+failure outcome. A complete regression and fresh clean blocking-soak check follow
+this checkpoint; neither is claimed here before it finishes.
+
+That follow-up completed at the `551a905` implementation checkpoint: full local
+regression passed 1,292 tests/126 suites with unchanged library scope/100% coverage,
+and both PR33373452034 and push33373448945 passed every matrix/lifecycle job.
+The single clean 30-second / 16-client gate recorded 4,368 sent, 4,363 received,
+five missing (99.88553114%), seven samples, all eight trends passing, empty final
+registries, final heap 99.909556% and handles 1→2. Raw report
+`coverage/soak-package-final-20260831.json` SHA-256:
+`4bd8bd5e95054da8c27f1236ca42335663fde6e303c3a850db3e4a684079b371`.
+This remains an unchanged-threshold short acceptance result, not lossless delivery,
+a new hour soak or an explanation of the exact earlier CI scheduling sequence.