flatsql 3.4.0 → 3.5.1

package/docs/PARTITION-STORE-WASM.md

@@ -26,11 +26,26 @@ departure from the design.
 
 - Create `env.memory` shared, with a maximum no larger than the import's
   (browser writer 256 MiB, reader 128 MiB, design §5.5).
-- Grow the heap before `flatsql_ps_start` when threads will allocate (alloc
-  then free a large block through `flatsql_ps_alloc`/`flatsql_ps_free`): V8
-  (Node 20–25, and browsers) refreshes each thread's view of a shared memory's
-  size lazily, so a thread touching memory another thread has just grown can
-  trap. WasmEdge is not affected.
+- **V8 hosts (Node, browsers): grow the heap to the memory's maximum before
+  `flatsql_ps_start`, so it never grows while guest threads run.** Allocate
+  through `flatsql_ps_alloc` in steps from 1 GiB down to 1 MiB until each
+  fails, then free them all (the allocator keeps the blocks; wasm memory never
+  shrinks; the guest allocator grows at most 2 GiB per call). V8 bounds-checks
+  `memory.fill`/`memory.copy` and atomics against each thread's cached view of
+  a shared memory's size, and another thread's `memory.grow` updates that view
+  only at this thread's next stack check. A thread that fills a block carved
+  from memory another thread has just grown then traps "memory access out of
+  bounds" (§4.3). A partial pre-grow only moves the failure: the 1,536 MiB the
+  test main used to grow was passed at 1,651 MiB in CI. Stop 16 MiB short of
+  4 GiB with a 4 GiB maximum: a heap segment ending at 2^32 wraps the
+  allocator's pointer arithmetic. `createPsNodeInstance` does this
+  (`growHeap: false` opts out, `heapBytes` caps it), and the Node host reports
+  `memoryGrownWhileThreadsRan` for every guest and warns when it is not 0.
+- **WasmEdge hosts do not pre-grow** (and SDN recycles reader instances on
+  their page count). WasmEdge keeps one page count shared by every thread's
+  instance and grows in place in a fixed reservation, so a grow is visible to
+  every thread at once. WasmEdge 0.16.4's AOT compiler must carry SDN's
+  `04-atomic-memarg-offset` patch (§4.3).
 - Give every guest thread a distinct `tid` below 2^29 (wasi-libc keeps it in
   mutex words).
 
@@ -93,8 +108,8 @@ budget fails the crash instead of making it partial.
 Added tests (`PS_SLOW_TEST`, run only when named): `parity_canonical_dump`,
 `parity_deterministic_bytes` (`cpp/test/ps/parity_test.cpp`) and
 `host_crash_ingest` / `host_crash_verify` (`host_crash_test.cpp`). The wasm
-test main grows the heap by 1,536 MiB before any thread starts
-(`PS_WASM_HEAP_MB`).
+test main grows the heap to the memory's maximum before any thread starts
+(§1; `PS_WASM_HEAP_MB` caps it).
 
 ```
 npm run build:wasm:ps-tests                        # artifact + test commands
@@ -163,6 +178,67 @@ journal. A negative control (half a data segment cut off) fails the verifier.
   when `mergeL0Blocks` exceeded the directory cap, so labeling, partition
   merges and acks stopped (`dac8d89`).
 
+### 4.3 The shared-memory grow race (V8) and AOT wait/notify (WasmEdge)
+
+**V8.** CI ps-wasm run 36603876540 (attempt 1) trapped "memory access out of
+bounds" in `writeTypeMergeOutputs` on the type-merge helper thread during
+`readers_under_saturating_writers_T2_1`, with memory at 1,651 MiB, past the
+test main's 1,536 MiB pre-grow. A Docker build with the Linux wasi-sdk 30 at
+the CI checkout path reproduces the CI test command exactly (its function
+indices match the trace, and its `flatsql-ps-threads.wasm` is the committed
+artifact's sha256). The trap offset `0x18d31a` is a `memory.fill`: the
+zero-fill of `blocks[i].resize(de.l0Len)` right after its `operator new`.
+Each thread's instance caches the memory size, and V8 bounds-checks
+`memory.fill`/`memory.copy` and atomics against that cache. Another thread's
+`memory.grow` updates the cache only at this thread's next stack check, so a
+block the allocator carves from memory another thread has just grown fails
+that check. Plain loads and stores are checked by guard pages and never trap.
+Measured on the Mac, Node 25.4.0, load 35–65:
+
+| Heap before threads start | T2_1 runs | Traps |
+|---|---|---|
+| none (`PS_WASM_HEAP_MB=0`) | 31 | 8, every one a `memory.fill`/`memory.copy` right after an allocation (vector push_back, string copy, the resize above) |
+| 2,000 MiB (past the test's peak) | 30 | 0 |
+| the memory's maximum (§1, now the default) | 500 | 0, and 0 runs grew memory after a thread started (1 run ended in a Node 25.4.0 SIGILL inside V8's JIT code, a host crash) |
+
+Natively the same test under ASan+UBSan (Docker linux/arm64, 4 CPUs, a CPU
+hog) found no memory error in 488 runs: the engine never overran.
+
+**WasmEdge, the SDN host.** Every thread's instance imports one shared
+`MemoryInstance`. On 64-bit hosts it is reserved in full up front (12 GiB,
+`lib/system/allocator.cpp`) and `memory.grow` maps pages in place, so the
+base never moves. The page count is one field that `memory.fill`/`copy`/
+`init` and atomic wait/notify check live, and plain loads and stores are
+checked by guard pages. The SDN C host I/O resolves every guest pointer per
+call (`WasmEdge_MemoryInstanceGetPointer`), and the Go accessor re-reads the
+page count on a miss. A grow is visible to every thread when the growing
+thread's `memory.grow` returns, and the guest allocator's lock orders it
+before any other thread can allocate from it. Measured: the same T2_1 with
+no pre-grow on the patched WasmEdge 0.16.4, AOT, had 0 traps in 26 completed
+runs.
+
+**WasmEdge AOT wait/notify.** Those WasmEdge runs also hung at teardown: 4 of
+30 with no pre-grow, 5 of 30 with a 1,536 MiB pre-grow. WasmEdge 0.16.4's AOT
+compiler passed `memory.atomic.notify` and `memory.atomic.wait32/64` the bare
+address operand and dropped the instruction's memarg offset; the interpreter
+adds it. wasi-libc addresses its thread-list lock as `i32.const 0;
+memory.atomic.notify offset=<lock>`, so those wakeups went to address 0 and
+thread exit and join hung. A plain spawn/join guest with no flatsql code, 50
+rounds of 16 threads, hung 11 of 20 runs AOT, 0 of 30 interpreted and 0 of 20
+under V8. `sleepNs` (a timed wait on a stack local whose frame offset the
+compiler folds into the memarg) compared another stack word and returned at
+once: 100 × 10 ms took 0.000 s AOT and 1.159 s interpreted. SDN's static
+WasmEdge carries patch `04-atomic-memarg-offset` from SDN `74db37e14`, and
+its substrate self-test checks it (tag `sdn3`, so threaded AOT artifacts
+recompile). With the patch, the plain guest ran 40 of 40, and T2_1 AOT ran
+140 times with no hang. Two of those runs failed. One ran out of the 4 GiB
+address space: the memio command keeps every file in guest memory, and
+passing runs end near 2.7 GiB. The other run's output was not kept.
+`memory_safety_sleep_and_notify_at_folded_offsets` passes. Without it, that test fails (20 of 20 folded-offset wakeups lost).
+`sleepNs` now passes its address through a volatile, so its wait has a zero
+memarg offset and sleeps even on an unpatched runtime. The artifact still has
+4 folded wait/notify sites, all in wasi-libc's thread code.
+
 ## 5. Design deviations
 
 1. **The Node host's thread spawn is its own pool, not the SDK's Node

package/docs/PARTITION-STORE.md

@@ -8,7 +8,13 @@ fan-out merge, admission and results. Part III (§23–§30) is T3: compaction,
 reclamation, meta-segment retirement, disk accounting and quota. Part IV
 (§31–§35) is T3b: store-migrate gseqs, the entry-size setting, hot-partition
 splitting and arrivals compaction; §36 records the quota eviction race T3 #3
-met on Linux. Part V (§37) bounds a reader instance's memory. The design is
+met on Linux. Part V (§37) bounds a reader instance's memory. Part VI (§38)
+records the memory-safety and race work: the ps-wasm trap, the sanitizers and
+the first disk-full emergency. Part VII (§39) closes the query gaps T6 found
+on the host-02 fixture and accepts records stored with their own size
+prefix. Part VIII (§40) makes a partition's bookkeeping O(1) or O(log S) per
+commit (the terabyte audit's B4) and adds manifest version 3 (M3). The
+design is
 the stack's `docs/architecture/flatsql-partition-store.md` (§22 overrides
 §0–§21; §22.4 holds the owner rulings). This file records what was built, how
 to run it, the measured acceptance, and every place the build departs from the
@@ -646,11 +652,11 @@ and idle-lane sub-cursors (§21 deviation 1).
 
 | Path | Contents |
 |---|---|
-| `ps/compaction.h`, `src/ps/compaction.cpp` | Manifest version 2, the compacted rows format, retire items; the compaction pipeline (plan, build, SWAP, apply, abort). |
-| `src/ps/reclaim.cpp` | The per-partition file ledger (`disk_bytes`), the RETIRE and UNLINKED records, reader-gated unlinking, meta-segment retirement (A9), open-time cleanup; the same for the type logs (meta log rotation and retirement, retired catalog runs and manifests, the type ledger). |
-| `ps/quota.h`, `src/ps/quota.cpp` | The quota planner, the space emergency, the ballast. |
+| `ps/compaction.h`, `src/ps/compaction.cpp` | Manifest versions 2 and 3 (§40), the compacted rows format, retire items; the compaction pipeline (plan, build, SWAP, apply, abort). |
+| `src/ps/reclaim.cpp` | The per-partition file ledger (`disk_bytes`; a hash table, §40), segment lookups by binary search, the RETIRE and UNLINKED records and the RETIRE valve, reader-gated unlinking, meta-segment retirement (A9), open-time cleanup; the same for the type logs (meta log rotation and retirement, retired catalog runs and manifests, the type ledger). |
+| `ps/quota.h`, `src/ps/quota.cpp` | The quota planner, the space emergency, the ballast; the planner's per-segment summary and the compaction candidate index, kept from a change feed (§40). |
 | `src/ps/sources-t3.cmake` | Registers the T3 sources and tests on the targets `cpp/CMakeLists.txt` defines. |
-| `cpp/test/ps/{compaction,orphan,quota}_test.cpp` | T3 tests. `checkPartitionDir` and `checkTypeDir` (fixtures) walk a partition's or a type's directory against the files its head and manifest name. |
+| `cpp/test/ps/{compaction,orphan,quota,bookkeeping_scale}_test.cpp` | T3 tests (the last: §40). `checkPartitionDir` and `checkTypeDir` (fixtures) walk a partition's or a type's directory against the files its head and manifest name. |
 
 ## 24. Compaction (§11, A11, minor 1)
 
@@ -693,7 +699,9 @@ in the manifest; rows' `seg` is rewritten); after a restart, when the
 partition's dead share is over the ratio and the per-segment counts are
 unknown, the oldest
 segment not yet surveyed (a build that would save under 10% is abandoned and
-its count remembered). `Engine::swapSegment` requests one (targeted, or the
+its count remembered). The owner keeps these candidates in an index updated
+from a change feed; a look does not walk the segments (§40).
+`Engine::swapSegment` requests one (targeted, or the
 oldest-generation compactable segment).
 
 Kill bound and readers. A type-level reader's visibility is `V_p =
@@ -733,10 +741,12 @@ appended.
   133 files in 0.31 s at a 0.3 s grace). BUSY (a handle is open) skips to
   later items and retries after 50 ms; idle
   reader lanes close cached handles after 5 s. The next batch carries
-  `UNLINKED` and the shrunken set; `disk_bytes` drops then. A set past 2,048
-  items unlinks its oldest regardless of the gate (a statement older than
-  hours of retirements gets the retryable SNAPSHOT_GONE instead of a writer
-  waiting on it).
+  `UNLINKED` and the shrunken set; `disk_bytes` drops then. The set rides one
+  ctl record of one batch, so it is held under a soft cap (2,048 items as the
+  record encodes them, less with a large lane table): past it the oldest go at
+  once, regardless of the gate and the grace (deviation 37). A statement older
+  than them gets the retryable SNAPSHOT_GONE; an open handle still makes the
+  unlink BUSY.
 - **Open** unlinks every file of the persisted set (a file a surviving reader
   instance still holds stays retired), the outputs of an INTENT_COMPACT
   without SWAP, and r/a files a first merge created without MERGE_DONE, then
@@ -1459,11 +1469,11 @@ harness's arrivals check ran on a second reader instance the reader gate did
 not cover, so a statement could lose a file to reclamation and end
 SNAPSHOT_GONE (on 2f6d86e too: seed 9003, trial 48); the gate now covers both.
 
-Open: `quota_disk_full_resumes_without_operator_T3_6` can stay in its first
+`quota_disk_full_resumes_without_operator_T3_6` could stay in its first
 emergency past the test's minute on a loaded box (one episode, three
-segments evicted, the ballast never back). It predates this change: on
+segments evicted, the ballast never back). It predated this change: on
 linux/amd64 Debug at a host load near 40, 4 of 100 runs on 2f6d86e and 1 of
-100 here; under ThreadSanitizer, 3 of 6 and 1 of 10.
+100 here; under ThreadSanitizer, 3 of 6 and 1 of 10. Root cause and fix: §38.
 
 # Part V: reader memory
 
@@ -1574,3 +1584,497 @@ held across statements by the reader beyond the budgets above):
 | writer: `L1Run::load` (`index_l1.cpp:168`) | every run's fences and all its blooms, held by `SegRun::run` for each warm partition's segments and each type's catalog runs | 1% of each run plus 1.25 B per entry | grows with the store: a terabyte store holds gigabytes in the writer instance; needs the reader's paging |
 | writer: `SegmentInfo::cdir` (`compaction.cpp:382`) | a compacted segment's presence directory, kept once loaded | 0.13 B per record | grows with the store |
 | writer: `Partition::segs`, `summary`, the file ledger | per segment | a few hundred bytes per segment | grows with segments |
+
+# Part VI: memory safety and races
+
+## 38. The ps-wasm trap, the sanitizers, the first emergency
+
+Task flatsql-ps-memory-safety-races-20260929. Machines: the owner's Mac
+Studio (Darwin 25.3.0, arm64, 28 hardware threads) shared with other lanes
+(1-minute load 17–95 during these runs), and linux/arm64 in Docker Desktop on
+it (gcc 13 and clang 18, `--cpus 4` with `stress-ng` on one CPU unless noted).
+
+**The ps-wasm trap is V8's shared-memory grow race, not an engine overrun**
+(PARTITION-STORE-WASM.md §4.3). CI's trap was a `memory.fill` right after an
+`operator new` on the type-merge helper, into memory another thread had just
+grown; V8 bounds-checks it against the thread's stale view of the memory's
+size. The wasm test main and `createPsNodeInstance` now grow the heap to the
+memory's maximum before any guest thread starts, and the Node host reports
+any growth after threads start. Natively, under ASan and UBSan, T2_1 found no
+memory error (loops below). UBSan found two defects:
+- A misaligned `uint64_t` reference: `Counters` is packed (on-disk).
+  `Partition::counters` is now 8-aligned.
+- Null-pointer arithmetic in `EntryIter::next` (index_l0.cpp). That fix
+  lands with flatsql-format2-query-gaps.
+
+Under WasmEdge the same runs exposed a WasmEdge 0.16.4 AOT defect: atomic
+wait and notify dropped the memarg offset. SDN's static runtime carries the
+fix, patch `04-atomic-memarg-offset` (SDN 74db37e14). `sleepNs` passes its
+address through a volatile, so its wait carries no memarg offset.
+`memory_safety_sleep_and_notify_at_folded_offsets` fails on a runtime without
+the fix.
+
+**ThreadSanitizer.** The whole default suite runs TSAN-clean (Docker, gcc 13
+`-fsanitize=thread`, `--cpus 6`) apart from the stage-1 races below. Each
+race found:
+
+| Where | What | Verdict, change |
+|---|---|---|
+| partition_log.cpp:2509 (`partitionPublish`) against type_owner.cpp:671 (`typeStage`) | The owner publishes a partition's commitSeq, pseqHi and L0 directory inside its seqlock; the type owner copies them and retries on a moved sequence. | Benign by design (a seqlock), but a plain copy on both sides is a data race in C++. Both sides now copy 8-byte words with relaxed atomic accesses (`publishPartition`, `readPublishedPartition`, writer.h). The sequence counter's fences order them. `PublishedPart::l0` is 8-aligned. `memory_safety_seqlock_publication_consistent`: readers racing a publishing owner only ever copy out whole states. |
+| writer_pool.cpp:230 (`SyncPool::runAll`) against `helpOne` | A pool thread reads a slot's `jobs` pointer while the writer publishes its next round; that stale pointer's claim then fails. | Benign by design. The pointer is `std::atomic` (relaxed); the round's `n` (release/acquire) orders the jobs. `memory_safety_sync_pool_rounds_run_each_job_once`: 6 writers × 400 rounds, each job runs exactly once with its own result, and the pool threads help. |
+| lane_arena.cpp:415 (`mxAlloc`) | Every lane's connection allocates SQLite's static mutexes again, which rewrote their shared wrappers. | Real (same values, rewritten under readers). The wrapper is filled once (`std::call_once`). |
+| lane.cpp:930 (`oldestActiveStart`) against `~ReaderInstance` | `Engine::setReaderGate` returned while a writer thread could still be inside the previous gate function, and the caller then destroyed the reader instance it reads. | Real: a use-after-free window. `setReaderGate` now clears the gate, waits for the gate calls in flight (a counter), then installs the new pair. |
+| merge.cpp:732 against open_test.cpp:296; reader_concurrency_test.cpp:216; compaction_test.cpp:565 | Tests read the owner's `nL0` from the main thread, or a latency series without its lock. | Test races. The tests read the published directory and take the lock. |
+| stage1.cpp:322 against stage1.cpp:499 (prep slots); ring.cpp:125 (a helper copies an entry the owner has consumed) | The owner republishes a slot it stole from a helper while the helper still reads it. | Benign by design (the helper's result is discarded by a failing CAS), and memory-safe (the copy stays inside the slab pool). Atomic slot fields and a racy-copy ring read belong to flatsql-ps-partition-bookkeeping-linear. |
+
+**T3 #6: the first emergency could stall for good** (§36). Traced under
+TSAN:
+- The eviction wave's partition had a full L0 directory (47 blocks) from
+  ingest before the emergency, so its TOMB_RANGE needed a merge first.
+- Meanwhile the room the ballast gave back was spent by maintenance outside
+  the eviction: the other partition's merges, and a compaction nobody
+  requested. Its build failed NOSPACE and started over with a new intent
+  batch in the meta log about ten times a second. Usage crept 8,960 B/s until
+  the device was exactly full.
+- From then on no batch committed. The wave's merge could not write outputs
+  and no TOMB step could stage, for the rest of the minute.
+
+36. **In a space emergency, maintenance leaves the ballast's room to the
+    eviction** (A13 assumes the ballast holds one segment's tombstones, and
+    says nothing of what else may spend it). Only a partition with eviction
+    work waiting merges (TOMB_RANGE steps, type-level kills, a requested
+    compaction: `partitionWantsMerge`). A compaction the planner did not
+    request does not start (`Writer::maintenance`); one in flight finishes.
+    Ring entries stop `kL0ControlSlots` (4) L0 slots before the directory's
+    cap, which mailbox kills and TOMB_RANGE steps may use, so a wave's first
+    steps never wait for a merge.
+
+Measured, in Docker:
+- **Before:** under TSAN, 4 of 12 runs, and 1 of 7 traced ones, stayed in
+  the first emergency until the test's limit (70 s).
+- **After:** 50 of 50 TSAN runs recovered (whole test 19–43 s), and 340 of
+  340 Debug runs (gcc `-O0`, 8–10 s each). The test also passes natively on
+  the Mac and under the Node host.
+
+**Loops** (Docker runs had a CPU hog on one of their CPUs; the Mac ran at
+load 17–95):
+
+| Item | Loop | Runs | Result |
+|---|---|---|---|
+| The trap | T2_1 under the Node host, heap grown to the maximum (Mac) | 500 | 0 traps. No run grew memory after a thread started. One run ended in a Node 25.4.0 SIGILL inside V8's JIT code (a host crash, not a guest trap). |
+| The trap | T2_1, native, ASan + UBSan | 488 before the T3 #6 change, then more on these sources | 0 ASan errors. UBSan reports only index_l0.cpp:319. |
+| The trap | T2_1 memio, WasmEdge 0.16.4 AOT with patch 04, no pre-grow (Mac) | 140 | 0 traps, 0 hangs. 2 failed: the memio command keeps every file in guest memory, and one run ran out of the 4 GiB address space (passing runs end near 2.7 GiB after about 500,000 records). The other run's output was not kept. |
+| Seqlock, sync pool | `memory_safety_s*` under TSAN | 520 | 0 warnings, 0 failures |
+| All races | The default suite under TSAN | 1 | Clean except the stage-1 races above. `arrivals_paging_while_writers_produce_T2_6` misses its 60 s label wait under TSAN, exactly as on 14a3075. `reader_point_memory_plateaus_under_commits` misses its idle-round bound under TSAN (0.155 MiB against 64 KiB). Both pass natively and under the Node host. |
+| T3 #6 | TSAN; Debug | 50; 340 | 0 stalls |
+
+**Writer memory that grows with the store.** §37's table lists the writer's
+per-store structures. Measured here with 200,000 OMM records in one partition
+at the defaults:
+- A warm partition's L1 accelerators (`SegmentInfo::runs`, loaded for every
+  segment by `partitionWarm`, merge.cpp:172) hold 6.0 B per record of the
+  partition's history. A partition written without a pause never cools
+  (`idleCloseMs`).
+- A type's catalog accelerators (`TypeOwner::runs`, loaded by `typeWarm`,
+  type_owner.cpp:580) hold 3.45 B per record of the type. A type owner never
+  cools.
+- Every registered partition keeps a 5,952-byte `Partition` (writer.h:505)
+  plus its ring descriptor, active or not.
+
+At 10^9 records (a terabyte of OMM-sized records) the catalog alone is
+3.4 GB in one writer instance, over the wasm32 limit.
+
+# Part VII: query gaps
+
+## 39. Query gaps found on the host-02 fixture (T6)
+
+T6 compared format 2 with format 1 on a host-02-sized migrated store (4.36 M
+records: OMM 1.70 M in one partition, MPE 1.69 M, CAT 141 k, IQC 419 k held by
+two producers). Five read shapes were slower than format 1, and store-migrate
+refused one record shape. Each is now an engine plan or check, held to a
+reference by `cpp/test/ps/query_gaps_test.cpp` (`query_gaps_*`) and to format 1
+by SDN's fixture comparison.
+
+**Text CID order (A17).** `ORDER BY _cid [DESC]` with no other index
+condition is its own plan (`kAccCidOrder`). At type level it reads the cid
+catalog (`kIxTypeCid`), whose keys are the A17 sort key: one ordered pass
+gives the live CIDs in text order. A key's entries resolve per copy (the
+latest type commit wins; DEAD over the rest at an equal tcs, as `catalog()`
+does); the live FIRST or PROMOTED copy is the record, and only its row is
+read. An `OFFSET` with no other condition is pushed down and counted from
+catalog entries alone: no row, frame or posting of a skipped record is read.
+At partition level the plan reads the partition's CID postings. With other
+conditions the plan still gives the order; the conditions are checked per
+row.
+
+**Per-object point profiles (A18).** Hidden input columns `_asof`,
+`_forward` and `_nearest` (epoch seconds, `=` only) select `kAccObjPoint`.
+Per object key it returns every live type-level row at the object's best
+second: `_asof` the latest second at or before T, `_forward` the earliest at
+or after, `_nearest` the closer of the two with a tie to the earlier. The
+plan reads the `OBJECT_EPOCH` postings `(object key, epoch)` of every
+partition, restricted by the manifests' epoch zone maps to the segments that
+can hold a candidate, merges the partitions by object key, and reads rows only
+for the candidates of each object's best seconds (from the closest outward,
+until one is live). Tag conditions and one `_epoch` range are applied before
+the choice; any other condition is refused (`SQLITE_CONSTRAINT`: SQLite would
+apply it after the choice, which is another query). `_object` projects the
+key: a u64 key in decimal, otherwise the (capped) string, NULL for a record
+without one. Records without an object key are read only from partitions whose
+`OBJECT_EPOCH` postings are fewer than their CID postings (the EPOCH index,
+their frames); each is its own entity. For OMM (NORAD, then OBJECT_ID) and MPE
+(ENTITY_ID) the object key is the legacy entity key, so SDN ranks the
+candidates exactly as format 1 did.
+
+**The sandbox window (A18).** Untrusted SQL reads `<TYPE>`, `<TYPE>@<source>`
+and every other type-level shape except a CID or gseq lookup from the arrivals
+tail: the newest N entries (`hotWindow`, dead entries included), the
+statement's conditions applied per row. An alias or source statement no longer
+walks every posting of its source; its work is bounded by N whatever the
+store's size.
+
+**Tag conditions on every copy (A2).** Type-level tag conditions (`_provider`,
+`_source_name`, `_batch`, `_peer_id`, `_source`, and the alias source) match a
+record when a live instance of any live copy (FIRST or REPEAT, any partition
+the plan reads) meets all of them, as the legacy tag table held every
+producer's tags per record. This extends §31.1 and T2 deviation 8.
+Row-driven plans check the FIRST copy's own tag first, then the other copies'
+own tags (listed by the cid catalog, only when the type has REPEAT postings),
+then RETAG instances. Posting-driven plans (tag, source, collected pages) let
+a live REPEAT copy's instance stand for its record: the record's FIRST row is
+emitted by the first matching copy in (FIRST, then REPEATs by pid, pseq)
+order, so each record comes once. `<TYPE>@<source>` is `<TYPE>` with that
+condition. The checks cost little per row: an instance's tuple is its lane's
+(`RecRow.laneId`, the partition's lane table read once per statement), a copy
+whose pseq is above the highest `max_pseq` of the lanes that meet the
+conditions cannot match (its RETAGs come after it), TAG_OF lookups read only
+the segments that can hold them and resume in the block the previous row's
+lookup read, and DEAD / TAG_DEAD lookups are skipped for a snapshot that holds
+no posting of the kind.
+
+**Gseq-ordered tag pages (A16).** At type level, `ORDER BY _gseq` with tag
+conditions and a LIMIT (or a closed gseq range) is served in arrivals order
+with each row's tags checked, unless the lane counters say the tags are rare
+(at most 262,144 matching instances and under an eighth of the type's
+arrivals): then the tags' postings are collected (the most selective of batch,
+source, provider, peer), sorted by gseq and emitted in order. A datasync page
+by source no longer sorts every match of the source.
+
+**Records stored with their own size prefix.** A record that carries its own
+size prefix (a `FinishSizePrefixed` buffer stored as is: SDN's
+dataset-publication PNMs and the local EPM) was refused with `kRejFid` (its
+identifier sits at frame offset 12), so store-migrate rejected 131 records of a
+dev node and refused to activate. `frameRootOffset()` (`extract.h`) finds the
+FlatBuffer after both prefixes (the record's u32 equals `len - 8` and the
+identifier is at 12) for the frame check, the verifier (at the record's own
+alignment origin), key extraction and column projection. The stored bytes and
+the CID (over the record, prefix included) are unchanged.
+
+**Reads ahead.** Rows met in ascending pseq order (arrivals order, epoch and
+tag postings, pseq scans) are read 32 at a time, and a frame read covers the
+frames of the rows read ahead in its segment (up to 256 KiB).
+`rowsExamined` still counts the rows a statement uses.
+
+**Measured** through SDN's storage API on the fixture (`TestZZBench`-style
+calls, each query twice in a fresh process: the first call pays the
+instance's warm-up and the page cache; format 1 is the same store before
+migration). Mac Studio (M3 Ultra, 28 cores), shared with other lanes, load
+47–55. Format 2 "before" is T6's run on the same fixture.
+
+| Gap | Query | Format 1 (call 1 / 2) | Format 2 before | Format 2 after (call 1 / 2) |
+|---|---|---|---|---|
+| 1 | OMM CID order, LIMIT 500 OFFSET 1000 | 730 ms / 26 ms | 9.9–28.4 s | 2.65 s / 3.6 ms |
+| 1 | CAT CID order, LIMIT 200 OFFSET 60000 | 2.39 s / 67 ms | 430–537 ms | 589 ms / 11 ms |
+| 1 | IQC CID order, LIMIT 1000 OFFSET 100000 | 3.90 s / 153 ms | 5.0–5.5 s | 4.14 s / 35 ms |
+| 2 | OMM `nearest` (32,015 objects) | 99 s / 22.7 s | 56–79 s | 10.6 s / 2.3 s |
+| 2 | OMM `as_of`, one source | 36 s / 19.5 s | 9.7–13.1 s | 3.9 s / 4.2 s |
+| 2 | OMM `forward` | 14.6 s / 19.5 s | 8.4–82 s | 6.8 s / 6.1 s |
+| 2 | MPE `nearest`, 2 h delta | 50 s / 17.6 s | 29–35 s | 7.1 s / 6.1 s |
+| 3 | sandbox `SELECT _data FROM "CAT@celestrak-satcat"` (N = 10,000) | 4.4 / 3.9 ms¹ | `timeout` (work budget) | 55 / 11 ms |
+| 3 | `"CAT@celestrak-satcat-csv"` | 1.3 / 1.3 ms¹ | `timeout` | 3.2 / 4.5 ms |
+| 3 | `"IQC@IQEngine"` | 14.8 / 13.3 ms¹ | `timeout` | 67 / 32 ms |
+| 3 | `"MPE@celestrak-gp"` | 3.2 / 4.3 ms¹ | `timeout` | 68 / 7.3 ms |
+| 4 | IQC source window, LIMIT 1000 (two producers) | 12.7 s / 2.0 s | 111–121 ms (per partition, merged in SDN) | 183 / 30 ms (one statement) |
+| 5 | IQC datasync by source, 5 pages of 500 | 133 s / 190 s | about 3.8 s a page | 58 / 32 ms (all 5) |
+
+¹ Format 1's hot window is an in-memory copy of the newest N records, filled
+by a hydration (17 s to 5.5 min on this fixture) and held in the arena whose
+growth is what format 2 replaces. Format 2 reads the window from disk: the
+engine's own cost for the CAT query is 2.4 ms natively (the rest is the wasm
+instance, the result ring and SDN's copy). Gap 3's acceptance (the statement
+answers within the work budget, bounded by N) is met; parity with the
+in-memory copy is not.
+
+The first call of a fresh process pays the instance's warm-up and reads the
+cid catalog's blocks into the page cache (2.65 s for OMM against 730 ms for
+format 1's index); every later call is faster than format 1.
+`TestFormat2DaemonReadsEqualTheLegacyFixture` (SDN) passes with every section
+(windows, A17 exports, A16 syncs, epoch profiles, A18 relations) equal to
+format 1 on this fixture.
+
+**store-migrate with size-prefixed records.** A copy of a dev node's store
+(9.2 GB control database, 163 PNMs in 37 partitions, 1 EPM): 0 copies
+rejected (131 before), partitions (58) and lanes (58) equal, 9 of 10 CID
+sequences equal. The tenth (IQC) differs because that store's index holds
+about 1.06 M IQC rows no producer table holds (orphans the legacy verifier's
+sequence includes; filed for SDN). `TestStoreMigrateActivatesWithSizePrefixedRecords`
+(SDN): a store with 12 PNMs in two partitions and an EPM migrates with 0
+rejects and 0 mismatches, activates, and serves the same bytes.
+
+**What these cost at 1 TB** (per statement; P partitions of the type, S
+segments per partition, R type catalog runs):
+
+- CID order: `(offset + limit) x copies per CID` catalog entries of about
+  72 B (about 56 per 4 KiB block) plus one row read per emitted record. The
+  skip is linear in the offset, as format 1's b-tree OFFSET is. Per-block live
+  counts in the catalog's fences would make it logarithmic, but a CID's copies
+  die in later type commits than their FIRST labels, so the type owner would
+  have to post a CID-keyed marker when a CID's last copy dies (outside this
+  change: `type_owner.cpp`).
+- Point profiles: the `OBJECT_EPOCH` postings (about 30 B each) of the
+  segments whose epoch zone can hold a candidate: for `_forward` near the
+  present, the newest segments; for `_asof` and `_nearest`, the segments
+  before (or around) T. No payload is read to rank; rows are read for about one
+  to two candidates per object. Enumerating the objects stays linear in the
+  postings of those segments: a per-partition object directory would make it
+  proportional to the objects.
+- Sandbox window: at most N arrivals entries (24 B) and N rows, whatever the
+  store's size.
+- Tag conditions: per row an O(1) lane check, and a TAG_OF lookup only for a
+  copy that a matching lane can still reach; a catalog lookup (R bloom-gated
+  runs) only in a type with REPEAT copies.
+- Gseq tag pages: LIMIT / selectivity arrival entries and rows per page
+  (arrivals mode), or the tags' matching instances (collected mode, at most
+  262,144).
+- Size-prefixed records: nothing (a 4-byte comparison per record).
+
+# Part VIII: writer bookkeeping at scale
+
+## 40. Per-partition bookkeeping at scale (TB audit B4, M3)
+
+The terabyte audit (stack `docs/architecture/flatsql-ps-terabyte.md`, B4)
+found a partition's bookkeeping growing as O(S²) in its segment count S. A
+single dominant partition (the GP archive import, a hot-split partition)
+reaches S = 1,000 at about 170 GB and S = 6,000 at about 1 TB. With 128 KiB
+seals, the rate fell from 116,829 rec/s at S = 235 to 465 rec/s at
+S = 5,188. Commit rounds reached a p99 of 100.7 ms and maintenance steps
+100.7 ms. The causes:
+
+- The ledger (§26) was a vector searched linearly.
+- `partitionPublishSummary` made three ledger scans per sealed segment on
+  every commit that carried a MERGE_DONE, SEAL, SWAP, RETIRE or UNLINKED.
+- The compaction look walked every segment every 50 ms, with the same three
+  scans and an L0 scan per segment.
+- Segment lookups were linear.
+- Open rebuilt the ledger in O(S²).
+- A MERGE_DONE and a SWAP recomputed the accelerator bytes by walking every
+  run.
+- A split partition re-signed its run set on every commit by walking every
+  run, and its helpers matched their previous view run by run in O(runs²).
+
+What the owner keeps now (`PartitionLedger`, `ps/quota.h`):
+
+- **The ledger** is an open-addressing table keyed by `(letter, seg, gen)`,
+  with the same matching rules as before: manifests by generation, `d r a m`
+  by segment, the rest by both. The byte total changes with every set and
+  drop. A lookup, change or drop costs O(1).
+- **A change feed.** A ledger change to a segment's files, a new entry, a
+  kill count, a survey and every input of a SWAP name the segments
+  concerned. The planner's summary and the compaction candidate index update
+  only those segments. The summary does this on the commits that publish
+  it, the index on each look. Open rebuilds both in one walk.
+- **The compaction candidate index** applies §24's rules unchanged:
+  - segments past the dead ratio, ordered by share (largest first, lowest
+    id among equals);
+  - starts of adjacent small pairs, where the first run of small segments
+    begins at the lowest pair and grows within the coalescing limits;
+  - segments not surveyed since a restart.
+  A look costs O(log S), plus the output's neighbours.
+- **Segment lookups** binary-search the ascending entry list. Entries are
+  created in order, and a pseq search steps over entries that hold no rows
+  yet. Open walks the unmerged sealed segments alongside the entries.
+- **Accelerator bytes** change by the runs a MERGE_DONE or SWAP adds and
+  removes.
+- **The split run set** is signed by the manifest generation, since every
+  change to it commits a new one. A republish for new L0 blocks shares the
+  run list instead of copying it. Helpers find reusable runs by
+  `(seg, gen)`.
+
+Stage-1 helpers (§32) no longer race the owner or producers (ThreadSanitizer):
+- A slot's `pos`, `len` and `kind` are relaxed atomics. The owner may take a
+  CLAIMED slot and republish it for the next ordinal while the helper still
+  reads them.
+- A helper copies an entry with `ringReadShared` (relaxed 8-byte word
+  loads), never through a pointer into the ring. After a steal, the owner may
+  consume and release the entry, and a producer may rewrite its bytes while
+  the helper copies.
+- `ringWrite` stores relaxed words and edge bytes, which natively cost what
+  plain stores do; the wasm hosts write slabs themselves.
+- In both cases the helper's CLAIMED to WRITING step fails and its result is
+  dropped, as before. What it parsed is its own copy.
+- Slabs must be whole 8-byte words.
+
+Deviations:
+
+37. **The RETIRE set is held under what one batch carries** (A12 said: past
+    2,048 items, the oldest go regardless of the gate once a grace has
+    passed). The set rides one ctl record, whose length is a u16, inside the
+    batch's 64 KiB ctl buffer, alongside UNLINKED (up to 1,024 items) and the
+    lane table. A set that does not fit refuses every batch of the
+    partition. A busy partition retires a manifest and folded runs per
+    merge, and a coalescing SWAP retires a dozen files per input, so within
+    the 60 s grace its set outgrew a batch. Ingest then stopped for the rest
+    of the grace. The audit harness showed 843 rec/s for 50,000 records at a
+    time, about every minute. The valve counts the set as its record
+    encodes it (consecutive manifest generations collapse). The soft cap is
+    2,048 items, or less when a large lane table leaves less room. Past it,
+    the oldest items are unlinked at once, without waiting for the reader
+    gate or the grace:
+    - A statement older than those items gets the retryable SNAPSHOT_GONE.
+    - A file held open is BUSY and waits as before.
+    - A meta segment still waits for a durable head (A9).
+    - Crash safety never depended on the grace. The batch that retired a
+      file is durable before its retirement counts, and open replays that
+      batch.
+    - While a compaction in flight pins the newest items and the set is past
+      halfway from the soft cap to the hard cap, merges wait. Merges are the
+      steady source of retirements, so the SWAP or abort can land first.
+38. **Manifest version 3 (M3)** carries the segment count as a u32 in the
+    header's last word (the u16 then reads 0xFFFF). It is written only when
+    a partition's list outgrows the u16, so an engine that predates it still
+    reads every store it can represent, and refuses a version 3 manifest
+    instead of misreading it. Every version now decodes strictly: the
+    records must end exactly at the CRC trailer. Before, a version 2
+    manifest over 65,535 segments wrapped its count, passed the CRC and lost
+    the rest.
+
+Measured on the Mac (Darwin 25.3.0, arm64, 28 hardware threads, shared) with
+the in-memory fault host. One partition, 128 KiB seals, coalescing off (so S
+is the seal count), 400 B pad records, lockStats on:
+`bookkeeping_scale_S5k`. The base column is 0df6ef4 (its writer is
+14a3075's) with the same test. Each row is the 50,000-record window that ends
+at that S.
+
+| | S = 235, base | S = 235 | S = 5,188, base | S = 5,188 |
+|---|---|---|---|---|
+| commit round p99 | 0.328 ms | 0.229 ms | 100.7 ms | 0.115 ms |
+| maintenance step p99 | 0.082 ms | 0.049 ms | 100.7 ms | 0.328 ms |
+| summary refresh p99 (per commit that publishes it) | | 0.016 ms | | 0.004 ms |
+| candidate look p99 | | 0.033 ms | | 0.012 ms |
+| open (half, full: S = 2,594, 5,188) | | | 45.7 ms, 170 ms | 4.0 ms, 4.5 ms |
+| rec/s | 116,829 | 99,796 | 465 | 5,386 |
+
+Load was 39–60 for the base run and 22–30 for the new one. The same test on
+this change over d566293, at load 26–37: commit p99 0.393 → 0.229 ms,
+maintenance p99 0.066 → 0.786 ms, summary refresh 0.012 → 0.020 ms,
+candidate look 0.049 → 0.016 ms, open 6.0 ms at S = 5,188, rate 77,556 →
+3,363 rec/s.
+
+Over the same 200,000 records (S = 0 to 940), the process used 5.9–6.9 s of
+CPU against 9.2–10.3 s on 14a3075 (six alternating runs). Near S = 235 the new
+build's wall rate was lower than base's by about 7% in alternating runs
+(82,405 against 94,237 rec/s, load 30–37), while at S ≈ 11 the two were equal.
+With shorter commit rounds, the writer commits smaller batches (about 940
+commits per 50,000 records against 700) and waits idle for the test's single
+producer. The time is lost in the handshake, not in work.
+
+On ext4, in Docker's Linux VM on the same Mac, the audit's harness
+(`tb_segments_scaling`: fsync on, one partition, 128 KiB seals, 550,000
+records) was bound by the VM's shared virtual disk. Other lanes' containers
+were using it too, so both builds ran near 500 rec/s at S = 235:
+
+| build | S = 235 | S = 1,886 | S = 2,594 | reopen at S = 2,594 | whole run |
+|---|---|---|---|---|---|
+| 14a3075 | 475 rec/s | 211 rec/s | 356 rec/s | 21.0 s | 31.3 min |
+| this build | 522 rec/s | 795 rec/s | 781 rec/s | 3.6 s | 19.0 min |
+
+`bookkeeping_scale_S5k` itself on the same ext4 volume (`--dir`) is disk
+bound: commit rounds wait on fsync (p99 336 ms at S = 235, 470 ms at
+S = 5,188). The bookkeeping in them stays flat, though: the summary refresh
+p99 is 0.041 → 0.033 ms and the candidate look p99 0.066 → 0.029 ms, and
+open at S = 5,188 takes 134 ms. With `--dir` the test reports commit and
+maintenance p99 without gating them.
+
+The rate that remains decays with S because of B5: every record's CID dedupe
+probes the bloom of every L1 run of every segment (`committedPostings`,
+`partition_log.cpp`). `sample` at S ≈ 4,900 puts 94% of the writer's samples
+in `cidState` and about 1% in the paths this section changed: maintenance,
+commit rounds and reclamation together account for 184 of 12,272 samples.
+Per-partition levelled lookup indexes (B5) are the terabyte design's.
+
+What still costs O(S), and where it belongs:
+- A merge snapshots every segment into the manifest it rewrites (M3's delta
+  or two-level manifest). The maintenance p99 above (0.05 → 0.33 ms) is
+  that.
+- A SWAP writes a whole manifest and rebuilds the entry list.
+- A split partition's helpers reopen every run file per run set (B2's
+  handle LRU).
+
+Tests (`cpp/test/ps/bookkeeping_scale_test.cpp`):
+- `book_manifest_v3_codec`: exact round trips at 0, 1, 65,535, 65,536 and
+  70,001 segments; a wrapped v2 count, a stray byte, a bad v3 header and
+  version 4 refused; version 1 read.
+- `book_retire_valve_keeps_ingest_flowing`: 60,000 records at the default
+  60 s grace. 16,767 files retired, 14,299 unlinked by the valve, the
+  encoded set peaked at 2,113, 0 batches refused. With the valve off, the
+  same test refused 4,397 batches and one 5,000-record chunk took 59.5 s.
+- The valve test also reports the writer's memory, and none of it grows
+  with records. From 25,000 to 60,000 records:
+  - slab pool: 3.9 → 4.1 MB;
+  - ledger: 2,271 → 2,398 files (67 → 69 KB);
+  - RETIRE set: 2,145 → 2,217 items (160 KB);
+  - segment list: 54 → 68 entries (12 → 15 KB).
+  The process does grow: 1.47 GB at 25,000 records and 3.55 GB at 60,000
+  (3.4 GiB of wasm's 4 GiB). That is the in-memory test host, which keeps
+  every file it ever wrote (7,212 files, 1.45 GB; then 17,281 files,
+  3.52 GB) so that data calls need no lock. The rest of the heap
+  (13 → 25 MiB) grows by the host's 231 B per file. On wasm the test runs
+  25,000 records.
+- `book_equivalence_all_commit_kinds`: runs with `gBookCheck`, which checks
+  every ledger operation against the linear ledger it replaced, and every
+  segment lookup, summary refresh, candidate pick and split run-set
+  signature against a walk. A mismatch aborts. The run covers seals,
+  merges, SWAPs (dead-ratio, coalescing, requested, quota evictions),
+  RETIRE and UNLINKED, meta-segment retirement, a hot split, a clean
+  reopen and a kill -9.
+- `book_crash_every_op` (every 37th mutating call) and `_full` (every one:
+  9,416 crash points over seals, merges, SWAPs, retirement, unlinking and
+  A9, all five crash modes, 0 failures): after each reopen, the rebuilt
+  ledger names exactly the files on disk at `disk_bytes`, counters equal a
+  recount, and the engine carries on under `gBookCheck`.
+- `book_check_fast_suite` (slow): every fast test again with `gBookCheck`
+  (88 of 88 passed).
+- `bookkeeping_scale_S5k` (slow): the table above. It gates commit and
+  maintenance p99 under 10 ms at S ≥ 5,000, and the summary refresh and
+  candidate look within 2× (+50 µs) of their S = 235 cost. On the base
+  commit it fails both p99 gates.
+
+Sanitizers, on this build over d566293 (macOS, clang):
+- ThreadSanitizer reports nothing on eleven tests: the four book tests,
+  `quota_arrival_order_heads_spared_T3_3`,
+  `compaction_cat_supersede_plateau_T3_2`,
+  `orphan_crash_points_during_compaction_T3_4`, and the four fast hot-split
+  tests.
+- AddressSanitizer with UBSan reports nothing on the book tests,
+  `compaction_removes_dead_rows_keeps_live`,
+  `quota_arrival_order_heads_spared_T3_3`, the orphan harness,
+  `hot_split_equals_unsplit_reference_T3_5` and `crash_faults_T1_1`. It had
+  found one defect in this section, fixed: `std::min` bound a reference to a
+  packed counter.
+- The stage-1 race, before and after, on Docker linux/arm64 with gcc 13 and
+  on macOS:
+  - 14a3075 reports it in 3 of 5 hot-split runs (`hot_split_throughput_T3_5`,
+    and `hot_split_rebalance_stalled_helpers_A26` at 12 s, twice);
+  - this change reports it in none of 6 runs.
+  - A two-thread program (`ringWrite` against a helper's read of the same
+    bytes) shows it on Linux with a plain reader, and still shows it with
+    atomic loads against a plain `memcpy` write. It is silent only once both
+    sides are atomic.
+- Under ASan the equivalence test needs more than its default 60 s to settle
+  (it passes with `--eq-settle-s=600`).
+
+Mutations the checks caught:
+- a missing kill touch;
+- a SWAP input not touched;
+- a ledger update dropped;
+- the valve off.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "flatsql",
-  "version": "3.4.0",
+  "version": "3.5.1",
   "description": "SQL queries over raw FlatBuffer storage: the SQLite-backed wasm engine and the multithreaded partition store (flatsql-ps-threads.wasm)",
   "type": "module",
   "main": "wasm/index.js",

package/wasm/integrity.json

@@ -5,9 +5,9 @@
   "size": 2246620,
   "files": {
     "flatsql-ps-threads.wasm": {
-      "sha256": "28f56226f97c16bc7e88c88c81003b0685a22a65f272e8cad7837ad380d0dd6d",
-      "sri": "sha384-uEQxf2jUYcPEQxCngoWcGlrAp2Fudkk/uK8OytYA4WRK+dp6S3D0HHlFd2maSwps",
-      "size": 2334491
+      "sha256": "3a215da45a53f3a7016257429c392720e728caf850d3a1213dc501bfb5844359",
+      "sri": "sha384-HJK5ECR2v2kAKnOHsyvXE+QgSywyuEUa2ipQQMhxR22dg7SZPAYPY985iyy2O7FP",
+      "size": 2485719
     },
     "flatsql-wasi-noeh.wasm": {
       "sha256": "8f11fd49ee2e6961b9c1c22dd891d5a6645d0faf481f815408c0f1f5d3a385ab",

package/wasm/ps-node-guest.mjs

@@ -144,6 +144,23 @@ if (mode === "pool") {
         reply(ptr);
       } else if (msg.op === "read") {
         reply(mem().slice(msg.ptr, msg.ptr + msg.len));
+      } else if (msg.op === "growHeap") {
+        // The heap to msg.limitBytes of memory before any guest thread starts
+        // (ps-node-host.mjs heapLimitBytes): allocate in steps from 1 GiB
+        // down to 1 MiB until each fails, then free them all. The guest
+        // allocator keeps the freed blocks; wasm memory never shrinks.
+        const { flatsql_ps_alloc: alloc, flatsql_ps_free: free } = instance.exports;
+        const held = [];
+        for (let step = 2 ** 30; step >= 2 ** 20 && held.length < 64; ) {
+          const p = memory.buffer.byteLength + step + 2 ** 20 <= msg.limitBytes ? alloc(step) : 0;
+          if (!p) {
+            step /= 2;
+            continue;
+          }
+          held.push(p);
+        }
+        for (const p of held) free(p);
+        reply(memory.buffer.byteLength);
       } else {
         throw new Error(`unknown op ${msg.op}`);
       }

package/wasm/ps-node-host.d.mts

@@ -11,6 +11,8 @@ export interface PsThreadReport {
   workerThreads: number;
   distinctWorkerThreadIds: number;
   memoryBytes: number;
+  /** Bytes the memory grew after the first guest thread ran (V8: a trap hazard). */
+  memoryGrownWhileThreadsRan: number;
 }
 
 export interface PsMount {
@@ -39,6 +41,13 @@ export interface PsHostOptions {
   deterministic?: { randomSeed?: bigint };
   initialPages?: number;
   maximumPages?: number;
+  /**
+   * createPsNodeInstance grows the guest heap to the memory's maximum before
+   * any guest thread starts (V8 can trap a thread touching memory another
+   * thread has just grown). false: no pre-grow. heapBytes caps the growth.
+   */
+  growHeap?: boolean;
+  heapBytes?: number;
   maxHandles?: number;
   instanceId?: number;
 }

package/wasm/ps-node-host.mjs

@@ -165,7 +165,27 @@ function createMemory(bytes, options) {
   }
   const initial = Math.max(limits.initial, options.initialPages ?? 0);
   const maximum = Math.min(limits.maximum ?? 65536, options.maximumPages ?? 65536);
-  return new WebAssembly.Memory({ initial, maximum, shared: true });
+  const memory = new WebAssembly.Memory({ initial, maximum, shared: true });
+  MAXIMUM_BYTES.set(memory, maximum * 65536);
+  return memory;
+}
+
+// The byte size each memory may grow to (createMemory).
+const MAXIMUM_BYTES = new WeakMap();
+
+// V8 bounds-checks memory.fill/copy and atomics against each thread's cached
+// view of a shared memory's size, and another thread's memory.grow updates
+// that view only at this thread's next stack check. A guest thread that
+// fills a block carved from memory another thread has just grown traps
+// ("memory access out of bounds"). So the heap grows to the memory's maximum
+// before any guest thread starts, and never grows after (a partial pre-grow
+// only moves the failure). The heap stops 16 MiB short of 4 GiB: a segment
+// ending at 2^32 would wrap the guest allocator's pointer arithmetic.
+// WasmEdge (the SDN host) keeps one shared size and is not affected.
+function heapLimitBytes(memory, options) {
+  const max = MAXIMUM_BYTES.get(memory) ?? memory.buffer.byteLength;
+  const cap = options.heapBytes ?? Infinity;
+  return Math.min(max, 2 ** 32 - 16 * 2 ** 20, memory.buffer.byteLength + cap);
 }
 
 /**
@@ -185,6 +205,9 @@ class Supervisor {
     this.listeners = new Set();
     this.stopped = false;
     this.poolErrors = [];
+    // Memory size when the first guest thread was seen running (tick): any
+    // growth after it is the precondition of the V8 stale-view trap above.
+    this.memoryAtFirstThread = 0;
     const warm = Math.min(options.warmThreads ?? 16, this.cap);
     for (let i = 0; i < warm; i += 1) this.startPoolWorker();
     this.timer = setInterval(() => this.tick(), 2);
@@ -208,6 +231,9 @@ class Supervisor {
     if (this.stopped) return;
     const want = Atomics.exchange(this.pool, POOL_WANT, 0);
     for (let i = 0; i < want; i += 1) this.startPoolWorker();
+    if (!this.memoryAtFirstThread && Atomics.load(this.pool, POOL_SPAWNED) > 0) {
+      this.memoryAtFirstThread = this.memory.buffer.byteLength;
+    }
     for (const listener of this.listeners) listener();
   }
 
@@ -242,6 +268,11 @@ class Supervisor {
       workerThreads: Atomics.load(c, CONTROL_STARTED),
       distinctWorkerThreadIds: tids.size,
       memoryBytes: this.memory.buffer.byteLength,
+      // Bytes the memory grew after the first guest thread ran (sampled every
+      // 2 ms). Under V8 that growth can trap a thread (heapLimitBytes).
+      memoryGrownWhileThreadsRan: this.memoryAtFirstThread
+        ? Math.max(0, this.memory.buffer.byteLength - this.memoryAtFirstThread)
+        : 0,
     };
   }
 
@@ -372,6 +403,14 @@ export async function createPsNodeInstance(wasm, options = {}) {
       reject(error);
     });
   });
+  // Before any guest thread: the heap to the memory's maximum (heapLimitBytes).
+  if (options.growHeap !== false) {
+    await new Promise((resolve, reject) => {
+      const id = nextId++;
+      pending.set(id, { resolve, reject });
+      worker.postMessage({ op: "growHeap", limitBytes: heapLimitBytes(memory, options), id });
+    });
+  }
   const rpc = (msg) => {
     if (failure) return Promise.reject(failure);
     const id = nextId++;
@@ -427,5 +466,11 @@ if (invokedDirectly) {
       `${t.spawnsDeclined ? `, ${t.spawnsDeclined} spawns declined` : ""}; memory ${(t.memoryBytes / 2 ** 20).toFixed(0)} MiB; ` +
       `${(r.elapsedMs / 1000).toFixed(1)} s\n`,
   );
+  if (t.memoryGrownWhileThreadsRan) {
+    process.stderr.write(
+      `ps-node-host: WARNING memory grew ${(t.memoryGrownWhileThreadsRan / 2 ** 20).toFixed(0)} MiB while guest ` +
+        `threads ran (V8 can trap a thread on memory another thread grew; grow the heap before threads start)\n`,
+    );
+  }
   process.exit(r.exitCode);
 }