redweb 0.12.0 → 0.13.0

package/docs/RECOVERY_FOLLOWUP_SPIKE.md

@@ -0,0 +1,147 @@
+# Deferred runtime-recovery investigation
+
+Status: deferred research, not a resolved defect or a recovery waiver.
+
+The maintainer subsequently authorized development, review and adoption of the
+server-focused check documented in `docs/SERVER_RECOVERY_CANDIDATE.md`. It now
+blocks CI; the original command remains unchanged as a visible non-blocking
+diagnostic. This changes the acceptance contract, not the historical results or
+the unresolved runtime explanation. No historical bounded comparison is repeated.
+
+The one-original/one-split Ubuntu comparison is now complete. See
+[its results and contrary ordinary-CI evidence](RECOVERY_COMPARISON.md).
+The stopping rule remains in effect: no repeat research measurement is needed.
+
+The maintainer prioritized finishing Redweb's release work over further open-ended
+runtime investigation. Exact V8 code-lifetime attribution is therefore a separate
+follow-up spike. It is no longer a prerequisite to finish the implementation.
+This does not by itself authorize shipping with a failed acceptance check.
+
+## What is parked
+
+- Explaining whether the attached Code objects in the private client snapshots
+  remain executable or are marked for invalidation.
+- Accounting for every hidden/runtime byte and the remaining unclassified parser.
+- Matching invalidated versions from the separate code-log run to snapshot objects.
+- Producing a minimal upstream V8/ws reproduction if the evidence warrants one.
+
+No more heap captures, compiler tracing, GC experiments or new diagnostic tooling
+are planned on the release path solely to answer these questions. Do not repeat
+the existing research or treat its partial explanations as a production fix.
+
+## Evidence already available
+
+[Offline attribution](RECOVERY_CODE_ATTRIBUTION.md) found 261 final
+deoptimization-bearing Code objects attached to preexisting functions/closures,
+one unchanged double-version group, and three unchanged standard-stream/prototype
+Socket objects. The [JIT control](RECOVERY_RUNTIME_CONTROLS.md) strongly supports
+a compilation-dependent contribution in the separate load-generator process.
+
+These findings do not establish the entire cause of the original shared-process
+Ubuntu/Node 22.23.2 failure. Its fourth storm reached 110.218742% against the 110%
+limit. One separately traced server also reached 110.080951%; server isolation is
+not assumed to guarantee a pass. Original failures and private evidence remain
+preserved. The snapshots must not be uploaded or embedded in a public issue.
+
+## Bounded release decision
+
+The next recovery-specific release step is a predeclared comparison using the
+existing original and split-process tools on Ubuntu/Node 22.23.2, with the matching
+client dependency identified. Record exact revisions, environments and every
+phase; keep connection counts, fixed warm baseline, delivery/cleanup checks and
+the 110% bound. No snapshots, tracing, JIT disabling, coverage instrumentation or
+other tests may run inside or alongside the measured processes.
+
+Use one original run and one baseline split run, sequentially, not retries until
+green. A failed setup is not a workload result. This is a bounded decision input,
+not enough by itself to establish repeatability or replace cross-runtime release
+checks. Separate processes change scheduling/GC and are not equivalent to the
+original shared-process measurement.
+
+Before execution, pin Node 22.23.2, record the Ubuntu 24.04 image/patch/kernel and
+any differences from the historical runner, and record source/lockfile/client
+hashes. Clear inherited diagnostic, coverage and workload overrides. Commands,
+in this order, with the second result collected even if the first fails, but only
+after confirming that the first process and all its children have terminated.
+If cleanup is uncertain, record the second run as blocked; do not launch it beside
+potential residual load:
+
+```text
+node --expose-gc scripts/verify-recovery.js
+node scripts/diagnostics/recovery-split.cjs baseline
+```
+
+After these two runs, stop, record both outcomes, and request the acceptance-contract
+decision. Do not automatically launch follow-up tracing or repeat either run.
+
+- If the isolated server satisfies its own budget and delivery/cleanup checks,
+  review whether an explicitly versioned server-focused acceptance protocol is
+  appropriate. Track client-generator health separately. Adoption is a separate
+  reviewed decision, not automatic promotion of this diagnostic or erasure of
+  historical failures.
+- If the server fails its budget, delivery, registry cleanup or process shutdown,
+  report the concrete failure and the required release decision. Do not start
+  another unbounded runtime-research cycle automatically.
+- A failed diagnostic, incomplete output or uncertain cleanup is not a pass.
+  The diagnostic's successful exit proves its stated delivery/cleanup conditions,
+  not that its reported heap ratios satisfy a release budget.
+
+Existing CI remains unchanged until a replacement protocol is explicitly approved.
+No threshold increase, selective rerun, code-byte subtraction or blanket
+`continue-on-error` follows from this deferral.
+
+### Temporary collection host
+
+The local Windows machine has no running Docker Linux engine or Ubuntu WSL
+distribution. A temporary, push-only `recovery-comparison.yml` workflow therefore
+collects the declared pair on a separate Ubuntu 24.04 hosted VM, pinned to Node
+22.23.2. It only triggers when that workflow file changes on `codex/agent-ready`,
+rejects repeated run attempts, and is removed after collection. It does not replace
+or modify `ci.yml`, and its completion does not adopt a new acceptance protocol.
+
+The historical failed runner reported Ubuntu 24.04.4, image `20260823.283.1`
+and runner agent 2.336.0 (run `33322376349`, job `99286494853`). The new run records
+its actual OS/image/kernel and Node/V8, revision, source/lockfile and client hashes.
+The historical log did not independently record its kernel. Matching the image
+label is not a claim of identical hardware or kernel.
+
+Each measured command runs in a transient systemd service with a minimal
+environment and `ExitType=cgroup`, so detached diagnostic worker groups remain
+owned. The 180-second outer cap and five-second termination bound do not change
+the scripts' existing internal deadlines. No resource quota or memory subtraction
+is added. Before either workload, two trivial real Node services exit 0 and 7
+through the same wrapper to verify status propagation and cleanup; these are setup
+probes, not recovery measurements. A failed command remains failed; an empty/removed cgroup must establish
+descendant termination before the split run starts. Systemd containment and clean
+environment differ from the historical direct-shell execution and are disclosed
+in the evidence. Other CI jobs use separate VMs, not necessarily exclusive physical
+hardware. Only explicit safe logs/reports/hashes are retained, never snapshots.
+
+The orchestration follows [systemd's transient-service semantics](https://raw.githubusercontent.com/systemd/systemd/v255/man/systemd-run.xml)
+and the kernel's [recursive cgroup population definition](https://docs.kernel.org/admin-guide/cgroup-v2.html#un-populated-notification).
+Its syntax/review and actual hosted execution are verification evidence, not a
+claim of unit coverage for shell orchestration. Both outcomes and the split server's
+actual warm/storm ratios still require review before the acceptance decision.
+
+## What remains release-critical
+
+- Actual message delivery, reconnect behavior, bounded queues and empty owned
+  registries after cleanup.
+- Successful shutdown checks; a reproduced timeout must be resolved or explicitly
+  dispositioned, not hidden by a longer deadline. The historical timeout remains
+  unexplained even when current regression tests pass.
+- Matching packed Redweb/client compatibility, required coverage and ordinary
+  test/load/memory/package checks. Existing gaps are not waived by this spike.
+- Honest documentation of the tested runtime, limits and remaining uncertainty.
+
+The full acceptance checklist remains in [AGENT_READY_ACCEPTANCE.md](AGENT_READY_ACCEPTANCE.md).
+This spike adds no new release feature, performance guarantee or certification.
+
+## Reopening the spike
+
+Resume only as deliberately scheduled work, or when a server-specific regression,
+continued growth under an approved protocol, or a real deployment incident makes
+these questions actionable. Start from the existing evidence and a single
+falsifiable question. Set the experiment count and stopping rule before executing;
+end with a reproducible cause/fix or a documented evidence boundary, not an
+indefinite sequence of deeper snapshots.

package/docs/RECOVERY_INVESTIGATION.md

@@ -0,0 +1,229 @@
+# Recovery investigation — 2026-08-30
+
+Status: unresolved. This investigation does not waive the fixed recovery gate.
+
+## Question and controls
+
+The existing five-cycle `steady-v2` gate intermittently exceeds its 110% retained
+heap limit despite empty connection, room and session registries. The retained
+earlier snapshots showed a temporary compiled-code increase, but did not prove a
+complete attribution or establish an application-owned leak.
+
+This investigation used Windows and Node 22.21.0, starting at Redweb `2410e60`.
+The workload remained 1,200 preconditioning connections, 200 warm connections,
+five rounds of 1,200 connections, batches of 50, and the existing 400 ms settling
+period and two explicit collections. No warm-up, threshold, dependency, V8
+acceptance flag or workload change was made. The local npm links were preserved.
+
+## Observations
+
+One separate `--trace-opt --trace-deopt` run showed repeated weak-object
+invalidation of compiled code in Node transport functions and Redweb's
+`requestSnapshot`, `decorate`, `prepareContext`, `ensureContext`, `handleConnection`
+and session paths. The captured console output was truncated, so it cannot support
+complete event counts or attribution. This trace is diagnostic only, even though
+the verifier's `diagnosticOnly` field does not detect Node trace flags. No raw heap
+snapshots were created. It does not prove that socket-property transitions caused
+the peak, nor that application retention is absent.
+
+Source review found avoidable per-connection allocations: a finite-capacity route
+without authentication or protocol policy still allocated an admission abort
+controller, pending-map record and promise chain. A capacity-only synchronous
+path was tested using the existing reservation and rejection logic without that extra
+admission lifecycle. Custom authorization and native `verifyClient` hooks remain
+on the existing asynchronous path. The route's connection-lifetime controller,
+when needed, was unchanged. No connection or exemplar object was kept alive to
+inflate the baseline or preserve V8 shapes.
+
+**That candidate was discarded.** Its eligibility check accessed an overridable
+authorization getter outside the protected promise path. It could also mistake a
+replacement on `SocketRoute.prototype` for the default implementation. Two added
+regressions failed on the candidate: the first threw synchronously instead of
+rejecting safely; the second bypassed a denial. Both pass on the restored original
+runtime. The initial reviewer approval was explicitly withdrawn after these
+findings. With no demonstrated recovery or throughput benefit, additional
+compatibility machinery would not justify keeping this optimization.
+
+## Unchanged acceptance measurements
+
+One ordinary run was taken before the change and one after it. These are individual
+measurements, not a repeated statistical comparison or a throughput claim.
+
+| Phase | Before: heap bytes | After: heap bytes |
+| --- | ---: | ---: |
+| Warm | 10,583,768 | 10,575,480 |
+| Storm 1 | 11,043,608 | 11,004,912 |
+| Storm 2 | 11,512,000 | 11,498,392 |
+| Storm 3 | 11,577,120 | 11,560,912 |
+| Storm 4 | 11,583,512 | 11,668,656 |
+| Storm 5 | 10,544,464 | 10,448,840 |
+
+The before run passed at a maximum 109.44601204410375%. The after run **failed**
+at 110.33689250984354%. All measured registries were zero in both runs. Final
+decline to 98.80251298286224% after the change does not waive the intermediate
+failure. No retry was used to replace that result. The allocation reduction is
+therefore **not a recovery fix**, and the previous CI failure remains relevant.
+
+## Functional verification
+
+The new real-listener regression failed before the change: a capacity-only
+connection still had a pending admission record. Afterward, the same regression
+verifies an active capacity reservation during connection, no asynchronous
+admission record, rejection at capacity, release on disconnect and successful
+reconnection. Separate real connections exercise custom authorization denial and
+asynchronous native verification. Unit cases exercise drain, destroyed transports,
+upgrade exceptions and reservation cleanup. These tests do not inject mocked
+transports into the integration cases.
+
+Before these extra compatibility cases were added, the discarded candidate
+passed 759 tests in 74 suites and all four instrumented-library coverage metrics
+at 100% (392.665 seconds), plus pretest/types. That broad pass did not detect the
+two subsequently demonstrated compatibility defects, and is not final-tree
+verification. The restored runtime with the two retained regression cases passes
+143 unit/real-network tests in ten suites, with all-four 100% coverage over
+`src/ws/BaseSocketServer.js`. Earlier narrower selections lacked unrelated
+constructor/inspection coverage and correctly failed their 100% coverage checks;
+their behavior tests passed. No threshold was changed.
+
+## Native WebSocket control
+
+After the full suite completed, one separate native-`ws` control acknowledged all
+7,400 messages using the same connection counts, batch size, settling delay and
+collection sequence. It loaded no Redweb server/runtime, but reused the existing
+native client open/close helpers. It omitted Redweb's rooms, sessions, authorization
+and application dispatch, so it is not a feature-equivalent benchmark or acceptance
+substitute. Its wire-response assertions and helper waits also differ from the
+acceptance driver. It measured zero native server clients after every phase.
+
+Warm heap was 6,919,096 bytes. Successive cycle heaps were 7,279,776 / 7,672,840 /
+7,755,872 / 7,819,640 / 7,829,720 bytes, ending at **113.16102565999951%** of warm.
+The control's exit 0 means its delivery/cleanup assertions passed, not that it
+passed a 110% memory gate; it deliberately reports measurements without enforcing
+Redweb's acceptance threshold. Its local source is retained at
+`coverage/recovery-native-control.cjs` (SHA-256
+`70484c74a59071fc8b9c73691c4e01dc6240abe65e15ea1a9e15536da03772e1`).
+
+This demonstrates that exceeding a 10% total-heap ratio is possible in this host's
+underlying network workload without Redweb. Together with the trace, it supports
+further investigation of runtime/compiled-code lifetime. It does **not** establish
+complete attribution of Redweb's failure or prove the absence of a Redweb leak.
+
+## Remaining investigation boundary
+
+The next recovery investigation must distinguish normal compiled-code lifetime
+from a concrete application retainer using a narrowly targeted observation. Do
+not subtract code bytes, retain artificial baseline objects, weaken the 110%
+limit, increase warm-up, or treat a later isolated pass as resolution. Publication,
+full original-client/application coverage and final cross-platform checks remain
+separate requirements.
+
+## Split-process milestone
+
+The original verifier measures Redweb, native client sockets and its load driver
+inside one Node heap. To distinguish those owners, the separate diagnostic
+`node scripts/diagnostics/recovery-split.cjs` starts a server worker and a native
+client worker. Only the coordinator retains earlier measurements. The acceptance
+verifier is unchanged; this diagnostic cannot pass or waive its heap gate.
+
+Both workers use Node's existing `--expose-gc` flag, without inherited Node options
+or coverage instrumentation. The server retains the original route, handler,
+room/session limits and timing. Work remains 1,200 preconditioning connections,
+200 warm connections, five 1,200-connection storms and batches of 50. Each batch
+waits for server connection cleanup. Each phase waits 400 ms, collects twice with
+one intervening immediate, and requires all measured registries to be empty.
+
+Differences are deliberate and material: there are separate runtimes, IPC and a
+coordinator between batches; replies are parsed and checked against their exact
+IDs; client operations use settled-result arrays and explicit ownership through
+failure cleanup. V8 statistics are read after the phase's heap measurement. These
+extra allocations and the changed scheduling mean ratios cannot be substituted
+for, added to, or directly compared as equivalents of the original shared heap.
+V8 spaces, code statistics, external memory and array buffers are overlapping views,
+not additive explanations of heap growth. No code-byte subtraction is allowed.
+
+Evidence is written to a new exclusive directory under `coverage/recovery-split-*`:
+complete per-phase JSON lines, a final JSON report, source fingerprints, versions,
+process identities, actual delivery counts, registry sizes and all memory views.
+Source fingerprints are checked again after completion. Output beyond 1 MiB is
+explicitly marked truncated. Each request and process cleanup is bounded; failed
+channels cannot be reused, and uncertain cleanup retains the PID and nested
+failures rather than claiming success. No raw heap snapshots are taken.
+
+### Single split run: observations
+
+Run once after the full suite finished, from 19:05:35.658 to 19:05:48.448 UTC on
+2026-08-30, Windows/Node 22.21.0, V8 `12.4.254.21-node.33`, native `ws` 8.21.3.
+The server PID was 22040; the load-generator PID was 34932. Both workers exited.
+All 7,400 request IDs were acknowledged exactly, and every phase had zero server
+connections/rooms/sessions and zero tracked client connections. Fingerprints were
+unchanged at completion. Neither worker emitted output; no output was truncated.
+
+| Phase | Server heap bytes | Client heap bytes |
+| --- | ---: | ---: |
+| Preconditioning | 10,181,120 | 6,579,816 |
+| Warm | 10,187,232 | 6,540,872 |
+| Storm 1 | 10,466,240 | 6,782,104 |
+| Storm 2 | 10,874,320 | 7,205,968 |
+| Storm 3 | 10,989,184 | 7,312,720 |
+| Storm 4 | 11,037,936 | 7,374,504 |
+| Storm 5 | 9,718,488 | 7,410,304 |
+
+The server peaked at **108.35068839111547%** of its own warm heap and ended at
+**95.39871085688438%**. The client crossed 110% at storm 2 and ended at
+**113.29229497229116%**, its highest measured value. This attributes the sustained
+above-110% ratio **in this split diagnostic** to the native load-generator process,
+which does not load Redweb's runtime or `redweb-client`. It does not establish the
+cause or relative contributions inside the original shared-process CI failure.
+
+The client's warm-to-final heap increase was 869,432 bytes. Its code-and-metadata
+statistic increased 521,972 bytes, while bytecode-and-metadata stayed at 755,952.
+Its old/code/trusted spaces increased 394,608 / 321,216 / 152,360 bytes respectively;
+external memory and array buffers remained unchanged after warm-up. These are
+overlapping views sampled sequentially, not an exact object-retention accounting.
+The data supports investigating client/runtime code and metadata lifetime, but
+does not establish that all growth is compiled code or exclude retained objects.
+
+On the server, bytecode-and-metadata fell from 908,240 bytes at storm 4 to 204,960
+at storm 5 while heap fell by 1,319,448 bytes. This is a correlated observation,
+not proof of a particular garbage-collector or bytecode-flushing cause. Server
+code-and-metadata remained 416,038 bytes above warm at the final measurement.
+
+Complete local evidence: `coverage/recovery-split-RQytod/report.json`, SHA-256
+`fa8b0a070dfb833a8449e4c658190a64778f9e425fed42c93251dda57ed35db7`, alongside
+`samples.ndjson`. The report includes all source fingerprints and memory views;
+the table above preserves every phase's heap measurement in version control.
+
+### Verification and next boundary
+
+The full suite passed **770 tests / 76 suites**, pretest/type checks, and all-four
+100% instrumented-library coverage in 427.392 seconds. Sixteen focused tests cover
+the diagnostic's fixed phase plan, fingerprints, nested errors, real HTTP-upgrade
+and WebSocket delivery, malformed replies, refused connections, timeout/exit,
+graceful and forced cleanup, and command validation. OS-cleanup uncertainty is
+tested separately with mocked units, not presented as real-process integration.
+
+The diagnostic scripts are **not at 100% coverage**. Native child-aware coverage
+reports 74.83% statements/lines, 95.12% branches and 80% functions across both
+scripts. Worker statements/lines are 97.36%; orchestration/CLI and failure paths
+remain uncovered. The earlier Jest-only report did not instrument child code and
+reported 36.72% statements; that was not a behavioral failure. Both coverage gates
+correctly exited nonzero without weakened thresholds; all sixteen behavior tests
+passed. Coverage instrumentation is explicit in behavioral tests and disabled in
+the actual measured workers. Library coverage does not certify diagnostic coverage.
+
+Final pretest/type/generated-documentation checks and 20 focused documentation/
+diagnostic tests also passed after the evidence update. The senior critic approved
+the milestone after independently checking report/source hashes, phase counts,
+registries and calculations; earlier cleanup/evidence findings were addressed.
+The next causal step is client-only retaining-path/code-lifetime
+inspection at warm and late storms, with complete bounded evidence, followed by a
+one-variable control. A speculative Redweb runtime optimization is not justified
+by this result. The original recovery failure, client/application coverage gaps
+and release requirements remain open. No original acceptance gate was rerun or
+waived, and no production code, dependency, threshold, publication or deployment
+changed in this milestone.
+
+Research-led follow-up: [runtime/JIT controls](RECOVERY_RUNTIME_CONTROLS.md) records
+the subsequent same-source baseline, complete trace and client-only JIT-disabled
+control. It narrows the client growth to a JIT-dependent effect without waiving
+the original recovery failure.

package/docs/RECOVERY_RUNTIME_CONTROLS.md

@@ -0,0 +1,181 @@
+# Recovery runtime controls — 2026-08-30
+
+Status: a strong JIT-dependent effect was identified in the split load generator.
+The original shared-process recovery failure is **not resolved or waived**.
+This continues [the split-process investigation](RECOVERY_INVESTIGATION.md).
+
+## Research and predeclared tests
+
+The V8 source bundled with Node 22.21.0 enables bytecode flushing with an aging
+threshold of six eligible collections. Forced and heap-profiler collections keep
+ages unchanged; the exposed `gc()` function requests a forced collection. Thus two
+explicit collections do not guarantee that unused code has aged enough to flush.
+These are implementation details of this exact runtime, not portable API promises.
+Sources: [flags](https://github.com/nodejs/node/blob/v22.21.0/deps/v8/src/flags/flag-definitions.h),
+[age handling](https://github.com/nodejs/node/blob/v22.21.0/deps/v8/src/heap/heap.h),
+[GC extension](https://github.com/nodejs/node/blob/v22.21.0/deps/v8/src/extensions/gc-extension.cc),
+[V8 bytecode-flushing explanation](https://v8.dev/blog/v8-lite).
+
+Three runs were declared before measurement, executed once each, sequentially:
+
+1. `node scripts/diagnostics/recovery-split.cjs baseline` — a fresh same-source control.
+2. `node scripts/diagnostics/recovery-split.cjs trace` — both workers use
+   `--trace-gc --trace-flush-code`, with additional sample markers.
+3. `node scripts/diagnostics/recovery-split.cjs client-jitless` — only the load
+   generator additionally uses `--jitless`; the server retains normal compilation.
+
+All use `--expose-gc`. No inherited Node options or coverage instrumentation enter
+the measured workers. Windows, Node 22.21.0, V8 12.4.254.21-node.33 and ws 8.21.3
+are unchanged. All three reports contain identical source fingerprints, also
+checked at each run's completion. No test suite ran alongside the measurements.
+The diagnostic changes no original acceptance script, workload, warm-up or budget.
+
+The fixed workload remains 1,200 preconditioning connections, 200 warm connections,
+five rounds of 1,200, batches of 50, connection-cleanup barriers and 400 ms settling
+with two explicit collections. Every run verified all 7,400 exact reply IDs and
+empty measured registries at every phase. Both workers exited in every run.
+
+## Measurements
+
+Heap bytes, captured before allocating each sample's subsequent V8 statistics:
+
+| Phase | Baseline server | Baseline client | Traced server | Traced client | JIT-control server | JIT-disabled client |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| Preconditioning | 10,171,000 | 6,568,792 | 10,174,984 | 6,571,376 | 10,186,408 | 5,566,904 |
+| Warm | 10,190,632 | 6,539,864 | 10,182,928 | 6,504,752 | 10,190,344 | 5,570,608 |
+| Storm 1 | 10,472,704 | 6,767,512 | 10,453,688 | 6,773,968 | 10,455,296 | 5,573,200 |
+| Storm 2 | 10,866,832 | 7,235,712 | 10,777,240 | 7,198,296 | 10,774,848 | 5,573,320 |
+| Storm 3 | 10,888,784 | 7,339,704 | 11,079,576 | 7,321,224 | 11,000,000 | 5,575,240 |
+| Storm 4 | 11,067,616 | 7,365,128 | 10,882,944 | 7,362,680 | 11,112,632 | 5,575,752 |
+| Storm 5 | 9,911,296 | 7,388,792 | 11,209,464 | 7,343,488 | 11,109,832 | 5,575,752 |
+
+The like-for-like comparison is baseline versus client-JIT control, **not** trace
+versus JIT control. Baseline client growth after warm-up was **848,928 bytes**
+(112.98082039626512% of warm); the JIT-disabled client grew **5,144 bytes**
+(100.09234180541873%). Baseline client code-and-metadata grew 504,608 bytes;
+the JIT-disabled client's corresponding value stayed exactly 2,058,560 bytes.
+Client bytecode-and-metadata, external memory and array buffers remained unchanged
+after warm-up in both runs.
+
+This is strong evidence of **JIT-dependent growth in this split workload**. It is
+not exact accounting of every retained byte: code/space statistics overlap, and
+disabling JIT also changes execution speed, allocation and collection behavior.
+The control additionally reports `Warning: disabling flag --expose_wasm due to
+conflicting flags`; that warning is preserved, not suppressed. The workload does
+not exercise WebAssembly. This is neither a throughput result nor a recommendation
+to disable JIT in production. See [V8's description of JIT-less execution](https://v8.dev/blog/jitless).
+
+The baseline server peaked at 108.60578617695154% and finished at
+97.25889424718703% of its warm heap. Its bytecode-and-metadata again fell late,
+from 908,584 to 205,248 bytes. That event **did not recur in the traced run**:
+server bytecode remained 908,584 and final heap reached 110.08095117632178%.
+The normal-JIT server paired with the JIT-disabled client peaked at
+109.05060712376344% and finished at 109.0231301318189%, also without the late
+bytecode drop. Retain all three observations; do not select the smallest result.
+
+## Complete trace and its framing limitation
+
+The traced server's stdout contains 12,979 bytes and the client's 12,257 bytes;
+both stderr files are empty. All file lengths and SHA-256 values were independently
+rechecked. Natural successful shutdown now drains output before exit; abrupt
+coordinator loss retains orphan protection and the coordinator retains bounded
+forced cleanup. Output is limited to 16 MiB per stream: overflow or a write failure
+invalidates the run instead of silently truncating successful evidence.
+
+V8's buffered native output and JavaScript sample markers interleave, occasionally
+inside a native line. **Raw marker adjacency is not a trustworthy timeline.**
+The original byte logs are preserved unchanged. For the counts below, remove only
+the fourteen exact `[rw-phase ...]` marker records (including their line endings)
+per worker from an in-memory copy. The reconstructed native records all have the
+expected process prefix and monotonically increasing V8 millisecond timestamps.
+No original bytes/files are overwritten, and no marker-based phase timing claim
+is made.
+
+| Reconstructed native records | Server | Client |
+| --- | ---: | ---: |
+| Scavenges | 37 | 41 |
+| Mark-Compacts | 19 | 16 |
+| Mark-Compacts labeled `testing` | 14 | 14 |
+| Mark-Compacts labeled `finalize incremental marking via task` | 5 | 2 |
+| Flushing summary records | 19 | 16 |
+| Total flushed SharedFunctionInfos reported | 0 | 0 |
+
+Server natural-major timestamps: 454 / 6,299 / 8,275 / 10,259 / 12,244 ms.
+Client natural-major timestamps: 1,323 / 4,201 ms. The fourteen testing collections
+per worker agree with two explicit calls for each of seven phases. The zero
+flushing totals and stable bytecode measurements are consistent with the V8 aging
+mechanism, but do not prove what caused the earlier untraced bytecode drop.
+No retry was made to obtain the desired flushing event.
+
+## Evidence locations
+
+Each directory contains `report.json`, `samples.ndjson`, and separate server/client
+stdout/stderr logs. Reports include flags, versions, identities, source hashes,
+per-phase memory views, delivery counts and output-file hashes. Times are UTC.
+
+| Mode | Local directory under `coverage/` | Start–end on 2026-08-30 |
+| --- | --- | --- |
+| Baseline | `recovery-split-RqlKQM` | 19:20:24.892–19:20:37.704 |
+| Trace | `recovery-split-bg0hV9` | 19:20:42.205–19:20:54.990 |
+| Client JIT-disabled | `recovery-split-z9JA5J` | 19:20:59.831–19:21:13.483 |
+
+Report SHA-256 values, respectively:
+
+- `212f24669b0240eff1680cb9aa91cef4690465f3203b6b81e6f8da50981d9a08`
+- `88fa6323feded06fea25f78169c88f79d7f4dcead997730d078203f8ee132f02`
+- `21b6c73bac8789c4f234363416df9fd740c0c7208b977e821f83c9377ccd7521`
+
+Trace stdout SHA-256 values:
+
+- Server: `29be4f852458b55af3ce6b77a9fd4b9b8213430c38e7c827c66231f2ae67d38c`
+- Client: `e006731ada264a63220e4dcafa6da080c049699e59c3f7caaabd2572cf14b4a2`
+
+## Verification and remaining work
+
+Twenty-four focused tests pass, combining unit tests with real child processes
+and real HTTP-upgrade/WebSocket traffic. New cases verify exact mode flags,
+complete output/hashes, cap failures, exclusive evidence creation and an 8 MiB
+queued-output shutdown fixture. That last test also passed before the drain fix
+on Windows; it must not be described as a reproduced POSIX truncation bug.
+
+Native child-aware diagnostic coverage remains below its unchanged 100% gate:
+76.03% statements/lines, 95.57% branches and 91.66% functions. The worker is at
+100% statements/lines/functions and 97.82% branches; its connection-barrier timeout
+branch remains uncovered. Coordinator/CLI and failure paths also remain uncovered.
+All behavior tests pass; the separate coverage command correctly exits nonzero.
+No exclusions or relaxed thresholds were introduced.
+
+Full-suite verification failed: **777 tests passed and one failed** across
+76 suites (75 passed, one failed), in 415.09 seconds. The failing case is
+`owned-http-shutdown.integration.test.js`: successful page cleanup with `live=true`
+timed out waiting 500 ms for the client's WebSocket close event, after application
+shutdown and incomplete-HTTP-peer closure had completed. Its cause remains
+unestablished; it is not dismissed as a timing flake or attributed to these
+diagnostics. No retry was used to replace this result. Pretest/generated-document
+and TypeScript gates passed. Instrumented-library coverage was 100% for statements,
+branches, functions and lines, but that does not override the behavioral failure
+or the separate diagnostic-tool coverage gaps above.
+
+Read-only critic inspection confirmed that the client close listener was installed
+before connection readiness, so no obvious missed-listener race explains the
+failure. Server shutdown bounds termination attempts but does not await the remote
+client's close notification. Endpoint state and error timing were not captured at
+the deadline; those observations are needed before attributing this failure.
+Final documentation generation/TypeScript checks passed, followed by 28 focused
+documentation and diagnostic tests across three suites. That scoped pass does not
+replace the failed full-suite result.
+
+The senior critic independently checked mode isolation, cleanup, report/log hashes,
+delivery/registry counts, reconstructed trace counts and the inference boundaries.
+There was no actionable evidence-integrity finding after the drain fix.
+
+The next causal investigation, if continued, should identify which compiled
+functions, feedback structures and retaining paths account for the client growth
+under normal JIT. The paired result does not establish an exact object-level cause
+or resolve the original shared-process CI failure. No further repeated benchmark
+is justified merely to obtain a smaller number or a flushing event. Original
+acceptance limits, runtime/library sources, dependencies, npm links, publication
+and deployment remain unchanged. No raw heap snapshots were taken.
+
+Follow-up: [client code-creation census](RECOVERY_CODE_CENSUS.md) identifies
+concrete normal-JIT compilation candidates without claiming retained-size proof.

package/docs/RELEASE_TRUST.md

@@ -0,0 +1,58 @@
+# Choose and verify a Redweb release
+
+Redweb is a Node.js HTTP/WebSocket library with server-rendered TSX, not a hosted service. Evaluate the exact package, runtime, application and deployment you intend to use. A passing test suite, a registry signature and a provenance statement answer different questions; none certifies an application as secure.
+
+## Runtime and compiler compatibility
+
+| Area | Contract and verification boundary |
+| --- | --- |
+| Core runtime | The package declares Node `>=18`. This is an installation/compatibility floor, not a recommendation to deploy an end-of-life runtime. |
+| Production Node | Use a currently maintained LTS release with current security patches. As checked on 2026-08-30, Node 22 and 24 are LTS; Node 18 and 20 are end-of-life. Recheck the official schedule when deploying. |
+| CI coverage | The repository matrix targets Node 18, 20, 22 and 24 on Linux. The 18/20 jobs are legacy-compatibility checks, not security-support claims. A configured job is not a passing result; inspect checks for your exact commit. |
+| TypeScript/TSX | The starter uses the package's tested TypeScript dependency and `redweb/tsconfig.json`. Standard decorators and legacy `experimentalDecorators` consumers have separate compile tests. Node's native TypeScript execution is not a replacement for compiling TSX/decorators with this configuration. |
+| Persistent dashboard | This application recipe requires Node 22.13+ and native `node:sqlite`; it is not part of the core runtime requirement. Its database and account/session design are single-process. |
+| Browser | The real-browser gates exercise Chromium. They are not a Firefox/Safari compatibility certification. Test the browsers you support, including reconnect and forms, before release. |
+| Runtime platforms | This branch has local Windows evidence and Linux CI configuration. Neither proves every OS, architecture, proxy, container platform or serverless host works. Live pages require a long-lived Node listener with WebSocket upgrades. Static export is a separate deployment mode. |
+
+Use the [official Node release schedule](https://nodejs.org/en/about/previous-releases) for maintained releases, the [TypeScript decorator documentation](https://www.typescriptlang.org/docs/handbook/release-notes/typescript-5-0.html#decorators) for the distinction between decorator modes, and [Node's TypeScript limitations](https://nodejs.org/api/typescript.html) for native execution constraints. Current per-commit test evidence and uncompleted checks are recorded in the [release checklist](AGENT_READY_ACCEPTANCE.md), not inferred from this table.
+
+## Pin the package and the documentation together
+
+For a published application, select an exact release, commit its lockfile, and use `npm ci` in CI/deployment. The following is an intentionally pinned example, not an instruction to treat `0.12.0` as permanently current:
+
+```sh
+npm view redweb@0.12.0 version engines dist.integrity dist.signatures dist.attestations gitHead --json
+npm install --save-exact redweb@0.12.0
+npm audit signatures
+npm audit --omit=dev
+```
+
+The signature command must run in the installed application directory. Keep TLS verification enabled and use a current npm CLI; a certificate/trust-store failure is not a reason to disable verification. A lockfile's integrity value detects changed package bytes; registry signatures authenticate registry metadata; provenance, when present and verified, links an artifact to a build/source identity. Vulnerability audit is a separate check against known advisories, not an application penetration test.
+
+For this branch's **unreleased** features, follow the matching tarball instructions in the [complete starter recipes](GETTING_STARTED.md#start-with-a-complete-recipe). The checkout has package metadata `0.13.0`, reserved for this unreleased candidate; the published `redweb@0.12.0` does not contain these changes. Installing `latest` does not install the unmerged branch. Do not mix a local-candidate guide with an older registry release and assume the APIs exist. Changing the package version does not publish it or satisfy outstanding release checks.
+
+Redweb is pre-1.0. Consult the changelog and versioned guide before upgrading, run your own real HTTP/WebSocket/browser tests, and keep a rollback artifact. Patch/minor numbers and a compatible TypeScript build alone do not prove wire compatibility, preserved sessions, database compatibility or application authorization. HTTP-created live-page sessions are process-owned; a restart or rolling replacement does not migrate them automatically. Raw socket protocol versions are negotiated only when the route opts in, and application payload compatibility remains your contract.
+
+## What was verified for the published package
+
+Read-only registry inspection on **2026-08-30** reported `latest: 0.12.0`, with:
+
+- `gitHead`: `6c95093c5e3b357772841f40c5371359dab0bf03`.
+- SHA-512 integrity: `sha512-74AOoEmREkoPElYB9nalsN8nLfiPzw1Ap3WYEwxERJqoSGcfYAQhyfpAK2dOyLQtc+A+sAl9fFv0HMuIXWDjEg==`.
+- Registry signatures present; **no `dist.attestations` field was returned**. Build provenance is therefore not claimed for this release.
+
+An independently created temporary application installed that exact package with lifecycle scripts disabled and ran `npm audit signatures --json` using Node 22.21.0/npm 11.6.2 and the Windows system trust store. It exited successfully with `invalid: []` and `missing: []`. That checks the installed registry dependency tree's signatures/available attestations; an empty failure list does **not** mean every package has build provenance. This historical observation is not a verification of a future release or of the current unreleased source.
+
+The commands above let you repeat the check. See npm's [signature and attestation verification](https://docs.npmjs.com/cli/v11/commands/npm-audit/) and [viewing provenance](https://docs.npmjs.com/viewing-package-provenance/) documentation for current verification behavior.
+
+## Publishing provenance is a maintainer action
+
+No publishing workflow or npm account setting is enabled by this guide. To add provenance to a future release, the maintainer must choose an authorized supported build/publish workflow, configure the correct repository identity and npm permissions, publish the exact tested artifact, and verify the resulting registry attestation afterward. A local `npm pack`, `gitHead`, checksum, badge or successful CI run is not a substitute for a verified attestation. Do not label older releases retroactively as provenance-verified.
+
+npm describes the supported providers and identity requirements in [generating provenance](https://docs.npmjs.com/generating-provenance-statements/) and [trusted publishing](https://docs.npmjs.com/trusted-publishers/). Provenance provides origin/build evidence, not proof that source code is safe.
+
+## Support and reporting boundaries
+
+For ordinary bugs, provide a minimal reproducible project, exact Redweb/Node/npm/TypeScript versions, operating system, decorator mode, sanitized logs, and the failing HTTP/WebSocket sequence in the [issue tracker](https://github.com/lakam99/redweb/issues). Never include tokens, cookies, passwords, private database contents or customer traffic.
+
+There is no paid support contract, response-time SLA or long-term backport policy established by these files. No private vulnerability contact is invented here. The repository inspection on 2026-08-30 reported no published security policy; the maintainer still needs to establish a private reporting channel and its handling policy before the project claims one. Do not disclose a suspected vulnerability or working exploit in a public bug report merely because that is the only linked tracker.