@harperfast/harper 5.3.0 → 5.3.1

package/resources/DESIGN.md

@@ -21,7 +21,7 @@ The Resource layer is Harper's universal abstraction: all queryable/mutable thin
 | `IterableEventQueue.ts`   | Async iterable used for subscriptions and streaming responses                                                                                                                                                                                     |
 | `transaction.ts`          | Per-request transaction object stored in `contextStorage`                                                                                                                                                                                         |
 | `auditStore.ts`           | Append-only audit log records                                                                                                                                                                                                                     |
-| `derivedIndexRuntime.ts`  | Lock-elected, exact-cursor delivery of committed RocksDB log mutations to derived-index backends; chunked collection, flush cadence, rebuild phase, epoch fencing, shared readiness, lag policy. Design: root `DESIGN.md` § Derived-index runtime |
+| `derivedIndexRuntime.ts`  | Lock-elected, exact-cursor delivery of committed RocksDB log mutations to derived-index backends; chunked collection, flush cadence, rebuild phase, epoch fencing, shared readiness, lag policy. See `indexes/DESIGN.md` § Derived-index runtime. |
 | `derivedIndexRegistry.ts` | Worker-local registration counts (which tables emit cache-eviction markers) and per-table write-admission checks for the lag policy                                                                                                               |
 | `indexes/fullText*`       | Bounded native Fulltext adapter, lazy capability-checked binding, and wrapper-owned inspection/open/reset/reclamation lifecycle                                                                                                                   |
 | `recordLock.ts`           | Exclusive record locks (harper#483): option contract, native key lock primitives (`lockAttemptKey`, `makeKeyLockHandle`, `acquireRecordKey`)                                                                                                      |
@@ -100,7 +100,7 @@ One giant `makeTable()` factory that returns a `TableResource extends Resource`
 | Where does versioning / conflict resolution happen?                            | `Table.ts → _writeUpdate` (`#section: write-path-internals`)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
 | How does `search()` choose an index?                                           | `Table.ts → search` (`#section: search-query`)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
 | How are subscriptions replayed?                                                | `Table.ts → subscribe` (`#section: pub-sub`)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
-| Can a saved audit cursor still catch up, or has its history been pruned?       | `auditStore.ts → getAuditFloor` — internal; there is deliberately no public accessor (harper#2458). **No resume path consumes it yet** — harper#2448 is to have `Table.subscribe` read it inside the resume, so the check and the replay cannot drift apart; until then a `startTime` below the floor is still silently truncated. One non-resume consumer does: `Table.commit`'s out-of-order reconciliation skips the audit walk for a write below the floor (harper#2642) — see "Audit retention floor" below. Returns the database-scoped floor: a cursor below it must resync, and `Infinity` means the floor is unknown (fails closed). `cursor >= floor` means only that no prune that ran _with a floor recorded_ removed history _after_ the cursor (nothing is promised below the FLOOR — that history is what a prune takes; `[floor, cursor)` is below the cursor but still covered). Two things it cannot see: history a legacy prune removed _before_ the floor existed, which a clock rollback can leave the stamped starting floor below; and a `restore_backup`/checkpoint rollback, since it is not a generation check (harper#2451). See "Audit retention floor" below.                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+| Can a saved audit cursor still catch up, or has its history been pruned?       | `auditStore.ts → getAuditFloor` — internal; there is deliberately no public accessor (harper#2458). **No resume path consumes it yet** — harper#2448 is to have `Table.subscribe` read it inside the resume, so the check and the replay cannot drift apart; until then a `startTime` below the floor is still silently truncated. One non-resume consumer does: `Table.commit`'s out-of-order reconciliation skips the audit walk for a write below the floor (harper#2642) — see ["Audit retention floor"](audit-retention.md#audit-retention-floor). Returns the database-scoped floor: a cursor below it must resync, and `Infinity` means the floor is unknown (fails closed). `cursor >= floor` means only that no prune that ran _with a floor recorded_ removed history _after_ the cursor (nothing is promised below the FLOOR — that history is what a prune takes; `[floor, cursor)` is below the cursor but still covered). Two things it cannot see: history a legacy prune removed _before_ the floor existed, which a clock rollback can leave the stamped starting floor below; and a `restore_backup`/checkpoint rollback, since it is not a generation check (harper#2451). See ["Audit retention floor"](audit-retention.md#audit-retention-floor).                                                                                                                                                                                                                                                                                                                                                                   |
 | How is the response body shaped (select clause)?                               | `Table.ts → transformEntryForSelect` (`#section: search-query`)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
 | Where is record-level TTL evaluated?                                           | `Table.ts → setTTLExpiration` (`#section: lifecycle-admin`); `Updatable.getExpiresAt` (`#section: setup-and-factory`). Stored expiry metadata is resolved in the `_writeUpdate` commit closure: `options.expiresAt ?? context.expiresAt ?? (record @expiresAt field, if finite & ≥ 0) ?? table default`. This metadata drives read-hiding + the cleanup sweep. The `@expiresAt` attribute is authoritative for **direct** put/patch only; cache/source fills persist via `recordUpdater` and derive expiry from `sourceContext.expiresAt` (source freshness / table default), not the field.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
 | Why does `search()` hide a row that's past its TTL but not yet swept?          | `Table.ts → transformEntryForSelect` unconditionally treats `entry.expiresAt < Date.now()` as gone (lazy eviction on read) — correct for a SELECT, but a mutation locating rows to overwrite needs the opposite: pass `target.includeExpired = true` (read by the SQL engine's `runUpdate`/`runDelete` via `SqlEngineContext.includeExpiredRows`) to treat such a row as a live match, matching the leniency a direct by-id `put`/`patch` already has (they skip this check entirely, since `Resource.patch`'s static options don't request `ensureLoaded`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
@@ -171,10 +171,6 @@ Consequences worth knowing:
 
 **Async false-mode read gates preserve the streaming contract.** `Table.search` returns an `ExtendedIterable` carrying the internal `SEARCH_AUTHORIZATION` promise. Static `Resource.search` and `query` await that verdict before returning a response; on success the wrapper initializes the real search before the transaction settles so its normal read snapshot stays reserved until iteration completes. The marker follows supported iterable transforms and retains `selectApplied`/`getColumns`, so async or mapped delegation cannot turn a denial into a truncated successful response.
 
-**Native query waits start on consumption.** A positive native `waitForIndexMilliseconds` preserves instance `Table.search`'s synchronous iterable result and normal `.map()`/`.concat()` composition. The custom-index adapter supplies a start gate: HNSW validates options and generation readiness synchronously, then waits for the first async pull before capturing its fixed coverage target. Zero-size pages start no native work. Count pages still materialize before returning.
-
-Index waits use the ordinary transaction timeout; they do not renew it. The adapter reuses the range-scan snapshot guard, loads through the captured read handle, and checks predicate reads so expiration cannot silently switch to latest-state reads. Iterator closure aborts pending waits and skips late result materialization. OR/concatenated prefixes may stream before a later branch fails. Waiting queries do not publish a coverage header; clients must consume the stream and check its error records. HTTP first-item status deferral is tracked separately in Harper #2670.
-
 **False-mode collection write gates stay per dispatch.** Built-in array PUT, query DELETE, and publish perform one request-scoped `allowUpdate`, `allowDelete`, or `allowCreate` verdict respectively. After query DELETE authorizes, it scans with a private cloned target whose permission check is disabled; the caller target stays untouched, and concurrent reads using it still run `allowRead`. Static publish overload routing marks the fresh per-dispatch resource receiver in `staticResourceDispatch.ts`, so copied targets and delayed delegation retain the `(target, message)` signature without putting reusable state on caller objects.
 
 **Array PUT is a collection dispatch in both modes.** `Class.put(batch, context)` arrives with no target, so `transactional` synthesizes one — and the collection it inferred from the null id has to carry over onto it, or the resource resolves as a single record with a null primary key and the batch never fans out. Default (instance) mode then dispatches per element through `getResource`, and each element call must carry its own normalized target in the second position — minted by `elementTargetFactory` so the element's id and `isCollection === false` sit on top of the request's query and route metadata, with `checkPermission` deliberately omitted so a per-element dispatch cannot re-arm the verdict the collection receiver already gave. Each element gets its own object, because its id must not be visible to a sibling whose dispatch resolves later; the request's contribution is resolved once for the batch rather than deep-cloned per element, and nested metadata is shared rather than copied — not the outer collection target and not the context: `Table`'s back-compat `put(target, record)` shift only recognizes a target that is a `RequestTarget`, so a context there is taken for the record and staged as record data (harper#2000). A default-mode `put()` override consequently sees one call per element with that element's id and `isCollection === false`; a component that needs the whole array in one call belongs in `loadAsInstance === false` mode.
@@ -185,143 +181,30 @@ Index waits use the ordinary transaction timeout; they do not renew it. The adap
 
 ---
 
-## Audit retention floor
-
-`Table.subscribe`'s `startTime` replay just begins wherever the audit log now begins, so a consumer
-resuming below the retention horizon is silently handed a short replay. The floor is the primitive
-that makes that detectable (harper#2447). It is internal, with deliberately no public accessor, and **no
-resume path consumes it yet**: harper#2448 is to put the check inside `Table.subscribe` itself — the same shape as
-replication's `shouldForceBaseCopyForRetention`, and the only one where the floor cannot move between
-being read and being acted on. Until then the short replay above is unchanged.
-
-**The one consumer today is not a resume**: `Table.commit`'s out-of-order reconciliation reads the
-floor before entering the audit walk (harper#2642). The walk terminates at the incoming write only by
-reaching an audit entry at or below its version, so below the floor it cannot — it runs the whole
-retained chain, one RocksDB end-of-log scan per step, to an outcome the floor already determines.
-Two things that consumer does differently from a cursor check, and both are deliberate: it compares
-the write's record version rather than a log key, because that is what the walk's own loop condition
-compares; and it treats the `Infinity` unknown floor as **walk anyway** rather than as "not safe",
-because here the conservative direction is to do the work, not to skip it. Below the floor a write
-contributes only its commutative operations — a plain field's survival depends on what newer writes
-did to that key, which is exactly what the pruned history no longer answers.
-
-**The invariant: every path that prunes audit history raises the floor BEFORE removing anything.**
-There are five, and the ordering is the whole guarantee — a floor written after the removal is lost
-if the process dies in between, and the surviving lower floor then certifies a cursor whose history
-is gone. Over-reporting (a floor covering more than the prune actually removed) costs a consumer one
-unnecessary resync; under-reporting loses its data with no signal. So `raiseAuditFloor` is called
-first and a throw from it is what stops the prune.
-
-| Prune path                                                                   | Engine                                                                      |
-| ---------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
-| `scheduleAuditCleanup` retention loop (`auditStore.ts`)                      | LMDB                                                                        |
-| `scheduleAuditCleanup` → `purgeLogs`                                         | RocksDB                                                                     |
-| `purgeAgedLogs` (boot/recovery, called from `replayLogs.ts`)                 | RocksDB                                                                     |
-| `Table.deleteHistory`                                                        | LMDB (`RocksTransactionLogStore.remove()` is a no-op, so it must NOT raise) |
-| `delete_transaction_logs_before` whole-database branch (`ResourceBridge.ts`) | RocksDB                                                                     |
-
-Things that are easy to get wrong here:
-
-- **The floor cannot be derived from the surviving log.** For four of the five paths the oldest
-  surviving entry would do, because they prune a database-wide time prefix. `Table.deleteHistory`
-  removes one table's entries from a database-scoped log, so a sibling's entry survives _below_ the
-  newest entry it removed, and a floor taken from that survivor certifies cursors over removed history.
-- **The record's presence is the trust marker.** `Symbol.for('audit-floor')` is a different key from
-  `last-removed`, which is still live and still maintained by the LMDB retention loop (#2338 hardened
-  its write path and added tests for the retry-carry — do not remove it). They coexist because they
-  answer different questions: `last-removed` records where the LMDB loop got to, after the fact,
-  while the floor is written ahead of every one of the five prune paths and its commit is verified.
-  A value found under `last-removed` therefore cannot be told apart from one carrying those
-  guarantees, which is why the floor needs its own key rather than reusing it.
-- **A store with no floor record is a store whose retention history we cannot account for.** That
-  includes the empty audit store an LMDB→RocksDB migration leaves behind, since `bin/copyDb.ts`
-  deliberately does not migrate it, and the audit-DBI-less result of a table-scoped backup taken
-  without `include_audit` — so `openAuditStore` stamps `max(Date.now(), newest retained key)` as a
-  one-time resync epoch. There is no permissive-baseline case: creating the audit DBI proves the
-  DBI was absent, not that the database is new.
-- **That epoch is a guess, and it is recorded as one.** Its bound is surviving state, which cannot see
-  history a selective prune already removed: a legacy `deleteHistory` takes one table's entries out of
-  the shared log, so a table that held the newest entries can leave the newest _survivor_ older than
-  entries that are gone, and a clock rolled back between the two stamps a floor below them (#2458).
-  Refusing to stamp is worse — `AUDIT_FLOOR_UNKNOWN` is absorbing (`raiseAuditFloor` cannot lift it,
-  `establishAuditFloor` skips any existing record), so it would make every upgraded deployment fail
-  closed forever. So `establishAuditFloor` writes the epoch under `Symbol.for('audit-floor-bootstrap')`
-  first, then stamps the floor from what that record holds.
-
-  **The record's presence is the signal; comparing it against the floor is not.** A store carrying one
-  has an unverified pre-tracking window for as long as the record exists, however far the floor has
-  since moved — a prune raising the floor above the epoch certifies only what that prune removed, and
-  says nothing about history removed before tracking began, which may sit _above_ the epoch, since that
-  is precisely what the guess could not see. Worked example: a v4-era `deleteHistory` removes tableA up
-  to t=1000 while sibling tableB's newest survivor is 900; a rolled-back clock stamps bootstrap=900 and
-  floor=900; a later retention pass raises the floor to 950. A repair keyed on `floor > bootstrap` would
-  read 950 > 900, call it earned, and leave a consumer at cursor 970 certified over tableA's missing
-  950–1000. So the mark is retired by a database generation (#2451), never by a floor that climbed past
-  it; what the recorded _value_ is for is telling that repair how far the guess reached.
-
-  Two properties it does depend on. **Ordering:** the record is written first, so a crash between the
-  two writes leaves a record with no floor, which the next open retries because the early return tests
-  the _floor_. **Undecodable bytes are overwritten** rather than kept — unlike the floor, where a
-  present record may be a deliberate `AUDIT_FLOOR_UNKNOWN` and rewriting it would lower a floor.
-  Keeping torn bytes pinned the store to unknown _forever_: the resolver skipped the write because a
-  record existed, the read back failed identically on every later open, and no retry could succeed.
-
-- **`getHistory` is not in the floor's time domain.** The floor is an audit-log key, which is what
-  `subscribe`'s events carry as `localTime`; `getHistory` reports each entry's origin `version` under
-  that same name, and a backdated or replicated write makes the two differ. A cursor saved from
-  `getHistory` cannot be compared against the floor.
-- **On RocksDB the floor tracks the configured retention horizon, not retained reality.** Whole-log-file
-  purge granularity means the branch cannot know which entries a purge will drop, and the floor is
-  written first, so each pass advances it to `Date.now() - auditRetention/(1+priority²)` whether a
-  file was dropped or not. Entries below that horizon are often still on disk, and a cursor among
-  them is told to resync — conservative in the safe direction only. LMDB can see a single eligible
-  entry, so it raises off the first one it finds instead.
-- **A prune's clamped floor can sit past `Date.now() + 1`.** `boundedAuditPruneEnd` records `newest + 1`
-  when the newest log key is at or past the clock, and log keys are fractional `getNextMonotonicTime`
-  values, not `Date.now()`, so for the rest of that key's millisecond the floor exceeds the wall clock
-  by more than one. Bound it by the newest key, never the wall clock alone (`auditFloor.test.js`, the
-  far-future `deleteHistory` case — it failed twice on main that way).
-- **Untrustworthy metadata resolves to `Infinity`, not to a number.** A wrong-length record, or eight
-  bytes decoding to NaN/negative, must not become a floor: `cursor < NaN` is false, so a consumer
-  spelling the check that way would read corrupt metadata as safe.
-- **A restore is outside what the floor can see.** `restore_backup` reinstalls the backup's floor
-  along with everything else, so a cursor from after the backup point reads as safe against it. The
-  audit floor is one of three carriers of resumable state a restore rolls back (record versions and
-  per-node `Symbol.for('seq')` records are the others), so this wants a database-level generation
-  rather than a fix in this one field — harper#2451.
+## Database generation and resumable positions
 
----
+Every copy path gives the copy a generation of its own before anything can read it (harper#2451), so no live subscription or persisted position carries over from the source. **Invariant: no readable copy carries its source's generation; a position resumes only if it names the current generation, its cursor is finite, and no prune within the generation reached above it** (`isResumablePosition`). RocksDB only: an LMDB database carries neither a generation nor a resume floor, so no position resumes against it, and a caller applies the check to RocksDB databases alone.
 
-## Path routing & parameterised routes
+- **Two records beside the floor:** the generation (a 16-byte random id and a float64 epoch, 0 for genesis) and the resume floor (highest prune cutoff in the generation). `raiseAuditFloor` raises both in one verified transaction; the resume floor is not absorbed by an unknown floor.
+- **Stamped into the copy before publication:** restore (before `completeRestore`), branch and migration (before their renames), via a private open plus an engine flush — a root-store write is not power-loss durable and directory fsync is best-effort. `harper copydb` and `compactOnStart` copy LMDB databases, so they stamp nothing.
+- **An ordinary open never repairs:** it adopts the record, mints genesis as a compare-and-set on absence, or leaves the handle without one (every resume refused). The resume floor is read as never below a finite audit floor, since an older binary's prunes raise only the audit floor.
+- **No scalar mode.** A position without an id is never resumable; bind an id only to a position established within the generation. Cursors must be progress-based: a snapshot's newest in-scope key can sit below a floor that retention advances on a quiet database, and would be refused forever.
+- **Live subscriptions:** the per-path registry outlives the store, but a subscription from before a reopen can never deliver again (its commit listener and table stores belong to the closed handle). Every open first records its audit store as the only handle a registration on the path may use, then detaches the registry and ends each subscription: `DatabaseGenerationChangedError` for another or an unknown generation on RocksDB, and otherwise (the same generation, or any LMDB reopen) the retryable `DatabaseClosingError`. A registration through an earlier or closed handle, even from inside that close, is refused with the same pair of errors.
+- **Not covered:** copies no generation-aware code made, a pre-generation binary pruning while the audit floor is unknown, keys reissued below a cursor after a clock rollback across a restart, cross-node identity (an id is per database per node), and subscription teardown and the handle check on a legacy LMDB `auditPath` root, which is reopened on every metadata read.
 
-`Resources.ts` is the registry that maps URL paths to `Resource` classes. Resources are registered (`jsResource.ts`) from a component's exports:
-
-- **Default path (convention):** the export name, resolved relative to the component's directory — `export class Widget` → `<dir>/Widget`.
-- **Declared path (`static path`):** a `static path` field overrides the convention. A leading `/` makes it root-relative (top-level); `./` or a bare name is relative to the component directory.
-- **Export-name-as-path:** `export { Widget as '/widget/:id' }` — the export name is the path (also honors the leading-slash root rule).
-
-A path is **parameterised** if any segment begins with `:` (named param) or `*` (wildcard/catch-all):
+---
 
-```ts
-export class Widget extends Resource {
-	static path = '/widget/:id/action/:action';
-	get(target) {
-		// GET /widget/10/action/jump → target.id === '10', target.action === 'jump'
-	}
-}
+## Path routing & parameterised routes
 
-export class Files extends Resource {
-	static path = '/files/*rest'; // GET /files/a/b/c.txt → target.rest === 'a/b/c.txt'
-}
-```
+`Resources.ts` is the registry that maps URL paths to `Resource` classes; `jsResource.ts` registers a component's exports. The user-facing rules — the default export-name path, `static path`, export-name-as-path, the root-relative leading `/`, `:name`/`*name` parameters and precedence — are documented in [Resources → Path Parameters](https://github.com/HarperFast/documentation/blob/main/reference/resources/overview.md#path-parameters).
 
 Mechanics:
 
-- **Registration** (`Resources.set`): parameterised paths are compiled into `paramRoutes` (kept _out_ of the base `Map`) so the exact/prefix matching fast path is untouched. Routes are ordered most-specific-first (more leading static segments, then longer patterns; wildcards rank last).
+- **Registration** (`Resources.set`): parameterised paths are compiled into `paramRoutes` (kept _out_ of the base `Map`) so the exact/prefix matching fast path is untouched. Routes are ordered most-specific-first: compared segment by segment from the left, a literal beats a `:param`, which beats a `*` wildcard (`SEGMENT_SPECIFICITY`); on a tie the longer pattern wins.
 - **Matching** (`Resources.getMatch`): exact and prefix matches are tried first and win ("static wins"); only when no static resource matches — and only if `paramRoutes` is non-empty — does `matchParamRoute` run. Matched segment values are decoded and stored on `entry.params`.
 - **Binding to the target:** request handlers (`server/REST.ts`, `server/DurableSubscriptionsSession.ts`) `Object.assign(target, entry.params)` after building the `RequestTarget`, so `:id` lands on `target.id`, `*rest` on `target.rest`, etc.
 - Named params match exactly one segment; a wildcard captures the remainder (zero or more segments) and must be the final segment.
-- **Discovery surfaces:** because parameterised routes live outside the base Map, the enumerators read `resources.paramRoutes` explicitly — `openApi.ts` emits them as templated paths (`:id` → `{id}`) with path parameters, and `components/mcp/resources.ts` lists them via `resources/templates/list` as `{param}` URI templates. `routePatternToTemplate` (exported from `Resources.ts`) is the shared `:param`/`*wildcard` → `{param}` converter.
+- **Discovery surfaces:** because parameterised routes live outside the base Map, the enumerators read `resources.paramRoutes` explicitly — `openApi.ts` emits them as templated paths (`:id` → `{id}`) with path parameters, and `components/mcp/resources.ts` lists them via `resources/templates/list` as `{param}` URI templates. The converter is not shared: `openApi.ts` uses `routePatternToTemplate` (`Resources.ts`), and `components/mcp/resources.ts` mirrors it in `paramPatternToUriTemplate` from the pattern string to stay free of a `Resources` import, so a change to the template syntax touches both.
 
 Tests: `../unitTests/resources/paramRoutes.test.js` (unit) and `../integrationTests/apiTests/param-routes.test.mjs` (end-to-end); enumeration coverage in `../unitTests/resources/openApi.test.js` and `../unitTests/components/mcp/resources.test.js`.
 
@@ -459,10 +342,9 @@ Blobs flagged with `saveBeforeCommit` (or `saveInRecord`) are written to disk in
 
 The mitigations live in three places:
 
-- `startPreCommitBlobsForRecord` (`blob.ts`) returns the blob list alongside its completion callback so each `TransactionWrite` can attach a `savedBlobs: Blob[]`.
-- `cleanupUnusedBlobs(blobs)` (`blob.ts`) waits for each blob's `saving` promise to settle, then `deleteBlob`s the file. It clears the input list so it's idempotent across repeated calls (e.g. an early-return that also gets caught by the abort path).
+- `startPreCommitBlobsForRecord` (`blob.ts`) returns the blob list alongside its completion callback so each `TransactionWrite` can attach a `savedBlobs: Blob[]`. `cleanupUnusedBlobs(blobs)` (`blob.ts`) waits for each blob's `saving` promise to settle, then `deleteBlob`s the file — it clears the input list so it's idempotent across repeated calls (e.g. an early-return that also gets caught by the abort path).
 - `Table.ts` commit handlers set `write.skipped = true` (and reset to `false` at the top of each invocation) on early-return paths that don't write the record/audit: duplicate-tie, superseded-by-put, no-audit-fullUpdate-loses, and cache-resolve version-changed. The transaction commit success paths (`DatabaseTransaction.commit` and `LMDBTransaction.commit`) walk writes and call `cleanupUnusedBlobs(write.savedBlobs)` for every still-skipped write. Cleanup is deferred (rather than run inline in the commit handler) because the commit handler runs again on optimistic-lock retries, and a retry can flip a previously-skipped write into a successful one (e.g. the existing record gets deleted between attempts so the older replicated update suddenly wins). Inline cleanup would race the deletion's `setTimeout` against the retry that referenced the blob.
-- `LMDBTransaction.abort` and `DatabaseTransaction.abort` walk all writes and run the same cleanup unconditionally (regardless of `skipped`), since nothing was committed. `DatabaseTransaction.commit` adds an explicit reject handler so a `Promise.all` failure on `completions` (e.g. a blob save errored) aborts the underlying transaction instead of leaking it _and_ the blob files.
+- `LMDBTransaction.abort` and `DatabaseTransaction.abort` walk all writes and run the same cleanup unconditionally (regardless of `skipped`), since nothing was committed. `DatabaseTransaction.commit` adds an explicit reject handler so a `Promise.all` failure on `completions` (e.g. a blob save errored) aborts the underlying transaction instead of leaking it _and_ the blob files. Both engines funnel synchronous throws and returned rejections from the whole commit chain through that cleanup once (`abortAfterCommitError`); LMDB captures `next` before its success path detaches the link, so a child's rejection still closes every uncommitted link. The cascade skips a link whose native commit is still outstanding — clearing its writes would unlink blobs that commit still references.
 
 **A blob instance outlives its file, so deletion tombstones it.** The `fileId` stays set on a `Blob` whose file has been unlinked, and `saveBlob` short-circuits on a set `fileId` — so a caller still holding the instance (the deploy recorder re-puts the same record object across a deploy) would re-encode a reference to a file that is gone. `cleanupUnusedBlobs` marks the blob's shared file state when transaction cleanup decides the file is no longer usable, before an in-flight save settles; slices share that state with their source. Ordinary supersession keeps its retention and re-reference window; reclamation marks every queued instance only after it claims the unlink. `saveBlob` throws on a discarded instance rather than minting a second reference (issue #2062).
 
@@ -478,17 +360,18 @@ Opt-in deflate compression for file-backed blobs (harper#2443) has three load-be
 
 `startMonitoringTxns()` (a `setInterval` per engine) watches `trackedTxns` and acts when a transaction's `timeout` reaches 0 (after ~2 ticks of `STORAGE_MAXTRANSACTIONOPENTIME`, default 30s). A transaction is tracked once it acquires a read snapshot (`getReadTxn`).
 
-- **Write-bearing request transactions are aborted and poisoned** (issue #1407). The monitor calls `abortDueToTimeout()`, which sets `timedOut`, forces `open = CLOSED` (so `doneReadTxn` takes the discard path instead of re-entering `commit()` via the `LINGERING` branch — which would now throw), then `abort()`s. `addWrite`/`commit` both guard on `timedOut` and throw `transactionOpenTooLongError` (503), so the in-flight request rolls back cleanly rather than the monitor silently force-committing a partial write set (atomicity violation + orphaned secondary-index entries that only a full rebuild repairs). The old behavior `commit()`d and reused the still-open transaction.
-- **`hasPendingWrites()` walks the `next` chain.** Writes to a second database live on `transaction.next` (see `txnForContext`), so a transaction that reads database A (head, tracked via its read snapshot, empty `writes`) and writes database B (`next`) is still write-bearing. Without the walk the head looks read-only and the monitor's force-commit path would cascade-commit B. `abortDueToTimeout()` poisons + aborts the whole chain.
-- **Read-only, `sourceApply`, and `isReplay` transactions keep the prior force-commit behavior.** Read-only long transactions (large scans/exports) have no atomicity/index risk and must not have their ongoing reads poisoned. Canonical-source applies (replication peer / external caching source) and crash-recovery replay have no resubscribe/resume path: aborting a write would drop it while the resume cursor advances past it — a permanent divergence (harper-pro#348). `sourceApply` is propagated down the `next` chain in `txnForContext`, so gating on the head suffices. (Replay is additionally synchronous, so the async monitor can't fire mid-replay anyway.)
-- **The owner's commit waits for the monitor's force-commit.** That commit claims every staged write, marks the transaction `CLOSED`, and detaches its handle, so the owner's later `commit()` finds nothing to do. The monitor therefore stores the promise it got back as `monitorCommit`: `commit()` chains on it and the monitor clears it on success. A failure is kept until the owner's final (`doneWriting`) commit rejects with it and releases the context, since the monitor itself only logs it. That release is deliberately not `abort()`: a multi-store commit can fail after its head store landed, and `abort()`'s blob cleanup would unlink files the head's audit entries still reference. A landed store always finishes its own bookkeeping (writes cleared, record locks released) before a chained store's failure surfaces, including a synchronous throw from `next.commit()`.
+- **Write-bearing request transactions are aborted and poisoned** (issue #1407). The monitor calls `abortDueToTimeout()`, which sets `timedOut`, forces `open = CLOSED` (so `doneReadTxn` takes the discard path instead of re-entering `commit()` via the `LINGERING` branch — which would now throw), then `abort()`s. `addWrite`/`commit` both guard on `timedOut` and throw `transactionOpenTooLongError` (503), so the in-flight request rolls back cleanly rather than the monitor silently force-committing a partial write set (atomicity violation + orphaned secondary-index entries that only a full rebuild repairs). The old behavior `commit()`d and reused the still-open transaction. `hasPendingWrites()` walks the `next` chain to decide this: writes to a second database live on `transaction.next` (see `txnForContext`), so a transaction that reads database A (head, tracked via its read snapshot, empty `writes`) and writes database B (`next`) is still write-bearing — without the walk the head looks read-only and the monitor's force-commit path would cascade-commit B, so `abortDueToTimeout()` poisons + aborts the whole chain instead. **Read-only, `sourceApply`, and `isReplay` transactions keep the prior force-commit behavior:** read-only long transactions (large scans/exports) have no atomicity/index risk and must not have their ongoing reads poisoned; canonical-source applies (replication peer / external caching source) and crash-recovery replay have no resubscribe/resume path, since aborting a write would drop it while the resume cursor advances past it — a permanent divergence (harper-pro#348). `sourceApply` is propagated down the `next` chain in `txnForContext`, so gating on the head suffices (replay is additionally synchronous, so the async monitor can't fire mid-replay anyway).
+- **The owner's commit waits for the monitor's force-commit, and a transaction parked in its commit phase is spared, not poisoned** (issue #2062). That commit claims every staged write, marks the transaction `CLOSED`, and detaches its handle, so the owner's later `commit()` finds nothing to do. The monitor therefore stores the promise it got back as `monitorCommit`: `commit()` chains on it and the monitor clears it on success. A failure is kept until the owner's final (`doneWriting`) commit rejects with it and releases the context, since the monitor itself only logs it. That release is deliberately not `abort()`: a multi-store commit can fail after its head store landed, and `abort()`'s blob cleanup would unlink files the head's audit entries still reference. A landed store always finishes its own bookkeeping (writes cleared, record locks released) before a chained store's failure surfaces, including a synchronous throw from `next.commit()`. Separately, `commit()` sets `committing` around its pre-commit await (the `before`/`beforeIntermediate` completions — in practice a blob's durable file write) and the monitor logs instead of aborting while it is set. The limit polices an _application_ holding a transaction open with an unfinished write set; once `commit()` is entered the write set is sealed and the caller is awaiting the commit, so the time is core's own I/O, and a multi-tens-of-MB deploy payload legitimately outruns the limit. Poisoning there was actively destructive: `abort()` cleared the write set and unlinked the write's pre-saved blobs, and the resumed commit then found nothing to write and resolved as **success** — the caller was told its write landed, and was left holding a blob whose file was gone but whose `fileId` was still set, so its next `put` silently minted a reference to a destroyed file (the deploy-payload case: `Blob file not found` on the peer, unrecoverably). The grace is bounded — `COMMIT_PHASE_GRACE` over-limit ticks, ~10 min at the 30s default, since sparing re-arms `timeout` — because the transaction still pins a read snapshot; a source that stalls rather than finishing falls through to the normal abort. `sourceApply`/`isReplay` are spared without a bound: they may be neither aborted (harper-pro#348) nor force-committed mid-write (that would durably commit a replica record whose blob file is still being written), and their blob sources are bounded by the receive-side idle watchdog instead.
+- **Resuming from that await re-checks that the transaction is still alive, but a submitted native commit is never aborted by the monitor.** `timedOut` (monitor poison, including via the `next` chain) throws `transactionOpenTooLongError`; a write set cleared with the handle released — a plain `abort()` in the same window — throws `Transaction was aborted while its commit was waiting on pre-commit work`. Without both, either path resolves as a phantom commit. `LMDBTransaction.commit` carries the same pair around its own `before` phase. Past the limit a submitted commit is deferred instead: after a short diagnostic grace the monitor poisons fresh ordinary follow-up work on the chain (`poisonAfterStalledSubmittedCommit`), reclaims read snapshots separate from the submitted writes, and logs once that the worker may need a restart. `sourceApply`/`isReplay` are observed, never poisoned. A pre-submit attempt on an already-poisoned link gets no commit-phase grace (`commitAttemptDoomedByPoison()`): its continuation throws on the resume check above, so sparing it would only pin its snapshot.
+
+**Extending the budget for one known-long write:** `DatabaseTransaction.timeoutBudget` is a per-transaction RocksDB floor applied whenever the transaction is re-armed (initial reads, writes, and active multi-store-chain propagation); the effective timeout is `Math.max(txnExpiration, timeoutBudget)`. This makes the budget sticky across a write's pre-commit existing-entry read and later writes, while never shortening a larger global `STORAGE_MAXTRANSACTIONOPENTIME`; RocksDB links added for another store inherit the same floor. Reads after a pending write do not re-arm the transaction: that preserves the idle-limit invariant for orphaned write-holding requests. Also, `resources/transaction.ts`'s `transaction(callback)` (no explicit context) joins whatever transaction is already open on the ambient AsyncLocalStorage context rather than guaranteeing a fresh one. `components/deploymentRecorder.ts`'s `withIsolatedTransaction` builds a new context from only the ambient audit/session fields — not the request `signal`, so a deploy client that stops waiting cannot cancel the tracking writes and strand an `hdb_deployment` row mid-phase (see the disconnect note below) — and every recorder write commits independently without inheriting transaction controls. It uses the sticky budget to give `ingestPayload`'s blob-gated writes a size-appropriate limit instead of the generic default, while coalesced progress flushes are drained and suppressed until ingest settles to avoid same-row transaction conflicts. The ingest helper deliberately floors the shared `deployment_timeout` at ten minutes because `0` means “poll once” for peer waits; consequently an ingest can pin its system-database snapshot for that minimum. Known gap (harper#2057): the extension only reaches RocksDB transactions — on `HARPER_STORAGE_ENGINE=lmdb`, `Table.txnForContext()` chains a separate `LMDBTransaction` (`txn.next`) with its own independently-reset timeout that the LMDB engine's monitor tracks instead. **A commit's conflict retries have their own deadline, separate from the open-transaction limit** (issue #2450). rocksdb-js ≥2.8 wakes a commit parked on another transaction's write intent after `ROCKSDB_JS_PARK_TIMEOUT_MS` (5s) and returns `RETRY_NOW_VALUE` even when the holder never releases, so a wedged intent presents as a stream of transient conflicts rather than one hung commit — and the `MAX_RETRIES` cap alone then keeps the request pending for ~40 park timeouts, minutes past the configured queue limit. `DatabaseTransaction.commitStartedAt` is stamped on the **chain root** at its first native submission and read at both retry decisions (the coordinated `RETRY_NOW_VALUE` resolve path and the `ERR_BUSY`/`ERR_TRY_AGAIN` rejection path); past `Math.max(STORAGE_MAXTRANSACTIONQUEUETIME, timeoutBudget)` the commit takes the existing `abortChainAfterRetries()` cleanup and throws a 503 `TransactionCommitConflictTimeoutError`. One clock per _logical_ commit, deliberately not per attempt: the per-attempt clock is `trackOutstandingCommit()`'s, which measures native liveness and drives `checkOverloaded()`'s thread-wide shedding, so back-dating it would let one uncapped `sourceApply` retry shed every unrelated request on the thread. The clock is released through the promise `commit()` returns (which settles only after the chained stores' commits), so a reused transaction's next batch starts fresh. `retryable` is true only on a chain root that has not rotated through a mid-scope commit — anywhere else an earlier store already wrote durable audit entries and ran its hooks that a replayed request would repeat. `sourceApply` is exempt, as it is from the attempt cap, for the harper-pro#348 divergence reason above.
 
-- **A transaction parked in its commit phase is spared, not poisoned** (issue #2062). `commit()` sets `committing` around its pre-commit await (the `before`/`beforeIntermediate` completions — in practice a blob's durable file write) and the monitor logs instead of aborting while it is set. The limit polices an _application_ holding a transaction open with an unfinished write set; once `commit()` is entered the write set is sealed and the caller is awaiting the commit, so the time is core's own I/O, and a multi-tens-of-MB deploy payload legitimately outruns the limit. Poisoning there was actively destructive: `abort()` cleared the write set and unlinked the write's pre-saved blobs, and the resumed commit then found nothing to write and resolved as **success** — the caller was told its write landed, and was left holding a blob whose file was gone but whose `fileId` was still set, so its next `put` silently minted a reference to a destroyed file (the deploy-payload case: `Blob file not found` on the peer, unrecoverably). The grace is bounded — `COMMIT_PHASE_GRACE` over-limit ticks, ~10 min at the 30s default, since sparing re-arms `timeout` — because the transaction still pins a read snapshot; a source that stalls rather than finishing falls through to the normal abort. `sourceApply`/`isReplay` are spared without a bound: they may be neither aborted (harper-pro#348) nor force-committed mid-write (that would durably commit a replica record whose blob file is still being written), and their blob sources are bounded by the receive-side idle watchdog instead.
-- **Resuming from that await re-checks that the transaction is still alive.** `timedOut` (monitor poison, including via the `next` chain) throws `transactionOpenTooLongError`; a write set cleared with the handle released — a plain `abort()` in the same window — throws `Transaction was aborted while its commit was waiting on pre-commit work`. Without both, either path resolves as a phantom commit. `LMDBTransaction.commit` carries the same pair around its own `before` phase.
+## A client disconnect cancels the request's writes (`resources/transaction.ts`)
 
-**Extending the budget for one known-long write:** `DatabaseTransaction.timeoutBudget` is a per-transaction RocksDB floor applied whenever the transaction is re-armed (initial reads, writes, and active multi-store-chain propagation); the effective timeout is `Math.max(txnExpiration, timeoutBudget)`. This makes the budget sticky across a write's pre-commit existing-entry read and later writes, while never shortening a larger global `STORAGE_MAXTRANSACTIONOPENTIME`; RocksDB links added for another store inherit the same floor. Reads after a pending write do not re-arm the transaction: that preserves the idle-limit invariant for orphaned write-holding requests. Also, `resources/transaction.ts`'s `transaction(callback)` (no explicit context) joins whatever transaction is already open on the ambient AsyncLocalStorage context rather than guaranteeing a fresh one. `components/deploymentRecorder.ts`'s `withIsolatedTransaction` builds a new context from only the ambient audit/session/cancellation fields, so every recorder write commits independently without inheriting transaction controls. It uses the sticky budget to give `ingestPayload`'s blob-gated writes a size-appropriate limit instead of the generic default, while coalesced progress flushes are drained and suppressed until ingest settles to avoid same-row transaction conflicts. The ingest helper deliberately floors the shared `deployment_timeout` at ten minutes because `0` means “poll once” for peer waits; consequently an ingest can pin its system-database snapshot for that minimum. Known gap (harper#2057): the extension only reaches RocksDB transactions — on `HARPER_STORAGE_ENGINE=lmdb`, `Table.txnForContext()` chains a separate `LMDBTransaction` (`txn.next`) with its own independently-reset timeout that the LMDB engine's monitor tracks instead.
+`transaction()` treats `context.signal` as the request's lifetime (harper#2001): once it aborts, the request stages no further writes, in this transaction or any later one opened on the same context — a static call joining a poisoned context transaction (`Resource.ts`), a fresh `transaction(context, cb)` on the aborted signal, and a retained instance writing after its scope completed all fail alike. Without this, an orphan keeps its staged writes and native write intents until the monitor sweeps it, and in the incident behind harper#2001 one held past a disconnect wedged a conflicting commit on the same worker. Work that must outlive the client runs on a context built without the signal (`components/deploymentRecorder.ts`); background work must not be armed inside a request's `AsyncLocalStorage` context for the same reason — the analytics flush is armed through `contextStorage.exit()` (`resources/analytics/write.ts`).
 
-**A commit's conflict retries have their own deadline, separate from the open-transaction limit** (issue #2450). rocksdb-js ≥2.8 wakes a commit parked on another transaction's write intent after `ROCKSDB_JS_PARK_TIMEOUT_MS` (5s) and returns `RETRY_NOW_VALUE` even when the holder never releases, so a wedged intent presents as a stream of transient conflicts rather than one hung commit — and the `MAX_RETRIES` cap alone then keeps the request pending for ~40 park timeouts, minutes past the configured queue limit. `DatabaseTransaction.commitStartedAt` is stamped on the **chain root** at its first native submission and read at both retry decisions (the coordinated `RETRY_NOW_VALUE` resolve path and the `ERR_BUSY`/`ERR_TRY_AGAIN` rejection path); past `Math.max(STORAGE_MAXTRANSACTIONQUEUETIME, timeoutBudget)` the commit takes the existing `abortChainAfterRetries()` cleanup and throws a 503 `TransactionCommitConflictTimeoutError`. One clock per _logical_ commit, deliberately not per attempt: the per-attempt clock is `trackOutstandingCommit()`'s, which measures native liveness and drives `checkOverloaded()`'s thread-wide shedding, so back-dating it would let one uncapped `sourceApply` retry shed every unrelated request on the thread. The clock is released through the promise `commit()` returns (which settles only after the chained stores' commits), so a reused transaction's next batch starts fresh. `retryable` is true only on a chain root that has not rotated through a mid-scope commit — anywhere else an earlier store already wrote durable audit entries and ran its hooks that a replayed request would repeat. `sourceApply` is exempt, as it is from the attempt cap, for the harper-pro#348 divergence reason above.
+- **Poisoning reuses the monitor's path, and cancellation is enforced where a write is admitted — but native submission, not entry into `commit()`, is the point of no destructive return.** `abortDueToDisconnect()` shares `abortAndPoison()` with `abortDueToTimeout()`: poison every link of the `next` chain, then abort each, and `addWrite`/`commit` throw `requestAbortedError` (499); `sourceApply` is never poisoned (no resume path, harper-pro#348). Every transaction created for a request carries its `signal` as `requestSignal` on the chain root — set by `transaction()` and by `txnForContext`'s per-context self-committing transaction. `admitRequestWrite()` runs in both engines' `addWrite` and in each deferred-save admission: an aborted signal poisons the chain and the write is refused; otherwise the first admitted write subscribes the chain root to the signal, and the listener is released once the chain's writes and commit attempts have settled — so a disconnect releases staged writes at once on every path that holds them, including a self-committing write parked in pre-commit work, while a read-only request registers no listener and always completes. A poison landing while a submitted attempt is in flight (`commitsInFlight` plus the chain root's submission marker) flags the chain but aborts nothing, so the attempt reaches its native outcome instead — aborting would race cleanup against an unknown outcome and unlink blobs a successful commit references (#2062), so a submitted write can land after the client is gone. For a disconnect the boundary covers the whole chain: once any store has submitted, the chain's unsubmitted stores ride the cascade already under way instead of being aborted, so the logical commit never splits across stores (#1407); the monitor still aborts an unsubmitted store independently, because it only reaches one whose pre-commit work outlived its grace.
+- **Transaction-owned iterators are tracked and closed when unreachable, and an abandoned scope keeps its handle to its own outcome.** `Table.search()` registers each result set with the transaction owning its read reference; `closeOwnedReadIterators()` runs their `onDone` across the chain. `transaction()`'s error path closes them (nothing was returned to consume them) and both monitors' `CLOSED` arm is the backstop, force-releasing a handle that closing does not return. A scope whose unawaited commit is still in flight when its callback throws is abandoned, not aborted (`abandonScope()`): the attempt keeps its handle and reaches its outcome, and the instance is closed and taken off the context so nothing joins it. **Coverage:** `Request`/`UwsRequest` supply the signal for HTTP and Node/Bun WebSockets, and `server/http.ts` drives the uWS WebSocket upgrade's signal from the adapter's `'close'`; MQTT contexts carry no `signal` (`server/DurableSubscriptionsSession.ts`), so MQTT is not covered. Tests use synthetic `AbortController`s (`unitTests/resources/txn-tracking.test.js` `Disconnect abort`, `unitTests/resources/transactionCommitBoundary.test.js`); no end-to-end socket test exists.
 
 ## RocksDB range activity and snapshot expiration
 
@@ -551,7 +434,7 @@ A failed transaction on a tagged stream (a non-retryable commit failure, includi
 
 A transaction can hold more than one write to the same record key — two `patch()` calls inside one `transaction()`, or a replicated transaction carrying two updates to a record. Each write captures `operation.entry` (its idea of the current record) when it is staged, and **neither engine can refresh that from a read**: LMDB queues staged puts and applies them only in the commit batch, so a `getEntry` inside that loop still returns the pre-transaction record (the exclusive `store.transaction()` fallback is no better), and RocksDB read-your-writes only sees writes already staged into the native transaction — which the source-apply path, staging its whole batch before `commit()`, hasn't done yet.
 
-So the writes carry the state forward themselves: `addWrite` chains each write to the preceding write to the same store and key (`linkWrite` → `operation.priorWrite`), a commit handler publishes what it stored on `operation.stagedEntry`, and the next write reads it back through `priorStagedEntry()` and uses it as `existingRecord`. Consequences worth knowing:
+So the writes carry the state forward themselves: `addWrite` chains each write to the preceding write to the same store and key (`linkWrite` → `operation.priorWrite`), a commit handler publishes what it stored on `operation.stagedEntry`, and the next write finds it through `priorStagedWrite()` (the nearest earlier write to the key that published a `stagedEntry`) and uses that entry as `existingRecord`. Consequences worth knowing:
 
 - Only the **record** comes from the earlier write. The rest of the entry (version, `localTime`/audit chain, blob metadata) stays the pre-transaction one — that is what this write's audit entry and optimistic version check are relative to.
 - Program order breaks **ties only** (`if (priorStaged && precedesExisting >= 0) precedesExisting = 1`). Every write in a transaction carries the transaction's single timestamp, so a version comparison against what the earlier write landed (e.g. on a retry round after a partial apply) is a tie that the out-of-order machinery would otherwise drop as a re-delivered duplicate. A _strictly newer_ existing version can only be a concurrent transaction's write observed on a retry round; a chained write still goes through the out-of-order merge for it (and a chained delete still yields to it) rather than silently overwriting it.
@@ -589,17 +472,13 @@ old value. The gap is general to `Table.save()`, not lock-specific. Exercised by
 
 ## A commit retry re-saves into the transaction it is retrying; `ImmediateTransaction.save()` must forward it
 
-Every retry and replay round of `commit()` (`ERR_BUSY`/`ERR_TRY_AGAIN`, `RETRY_NOW`, the
-retained-iterator replay) passes its native transaction into the save loop so each re-save re-stages
-into the handle being retried. `ImmediateTransaction.save()`'s `isCommitting` branch is the one
-`save` that could drop that argument: with none, `DatabaseTransaction.save()` opens a fresh handle and,
-the wrapper being `CLOSED` by then, commits it through a nested `commit()` whose own retry loop
-(`retries > 0` skips nothing) re-enters the override — unbounded, synchronously (`RangeError` on the
-first conflicting hold-lock save of a request). The override forwards the handle; the named test
-`immediateTransactionConflictRetry.test.js` asserts two commit attempts on one native transaction id.
-A retry round's handle is not `this.transaction` (detached before the first submission), so a throw
-from the re-save loop — a lapsed lease refusing the re-save — releases it explicitly before `abort()`,
-or its write intents park other writers until GC.
+Every retry and replay round of `commit()` re-stages into the native handle being retried, and
+`ImmediateTransaction.save()`'s `isCommitting` branch forwards that handle: without it,
+`DatabaseTransaction.save()` opens a fresh handle and nests a `commit()` whose retry loop re-enters
+the override without bound (`RangeError`, harper#2825). A retry round's handle is detached, so a
+re-save that throws releases it before `abort()`, or its write intents park other writers until GC.
+Pinned by `immediateTransactionConflictRetry.test.js` ("…not a nested commit", "releases the replay
+transaction when a lapsed lease refuses a re-save…").
 
 ## A transaction is joinable as a scope only if it stages its writes (`transaction`/`Resource`/`Table`)
 
@@ -659,7 +538,7 @@ It has to be one codec, decided before the first open, because RocksDB opens **e
 
 Opens pass `compressionForAllColumnFamilies` (rocksdb-js) alongside the codec. Without it the binding gives every family the caller did not name its _persisted_ algorithm and applies the request only to the target, so families this process never names individually would keep their original codec forever — which is why a database created before the prebuild carried any codecs (every 5.1 instance) stayed uncompressed no matter how new the binary was, and why reconciling a table afterwards failed with `already open with compression ...; cannot reopen it with ...`.
 
-The ordering that still has to hold is a fresh install: `install()` calls `mountHdb()` — which creates the system families — several steps before `createConfigFile()` writes the config file, so `installer.ts` stages the value into the in-memory config (`stageRocksCompression()`, mirroring the `STORAGE_ENGINE` line beside it) before `mountHdb()` runs. Measured without it, same pid: `thread=0 resolved=undefined`, `thread=1/2 resolved=zstd`, and the boot dies with "The system database failed to load".
+The ordering that still has to hold is a fresh install: `install()` calls `mountHdb()` — which creates the system families — several steps before `createConfigFile()` writes the config file, so `installer.ts` stages the value into the in-memory config (`stageRocksCompression()`, mirroring the `STORAGE_ENGINE` line beside it) before `mountHdb()` runs. Without it the main thread resolves no codec while workers resolve the configured one, and the boot dies with "The system database failed to load".
 
 Changing the codec governs newly written files. Existing SST/blob files keep theirs until rewritten; ordinary compaction will not do it (RocksDB skips the bottommost level without a compaction filter), so converting an existing database in place needs `compact({ bottommost: true })`.
 
@@ -704,129 +583,18 @@ The additive-only rule above repairs the _consumer_ of a partial peer snapshot;
 
 ## Subscription version selection (`Table.ts`)
 
-`Table.subscribe` resolves audit events through `eventFromAudit` for both replay and live delivery: unless `includeSuperseded` (defaulting to `rawEvents`) is enabled, record mutations must match the current primary version; an absent primary entry cannot establish a current version and is skipped. Publishing advances that primary version too: a put followed by a publish replays the message but not the put under the default rule. Messages and control events are independent. Version filtering precedes historical reconstruction and `previousCount` acceptance, while audit cursors advance over rejected entries. Single-record replay retains its bounded history walk because it also contains independent published messages; only an accepted replay event suppresses the current snapshot, and `eventFilter` also applies to that fallback. Default replay reads the current primary entry for each mutation. Opting into superseded full records retains audit-history reconstruction costs and retention limits, including patch reconstruction on durable MQTT live delivery. Collection `startTime` replay leaves `subscription.startTime` — the exclusive resume cursor `DurableSubscriptionsSession.saveSubscriptions` persists — never past a key with an unhandled in-table record: RocksDB gives every record of a transaction one `txnLogKey`, so the cursor moves to a key only on reaching the next key or finishing, and an early return (failed send, close at a yield or drain) leaves it up to one transaction back, which the resume re-sends (`subscriptionReplay.test.js`, "cursor after replay stops early"). The cursor counts events handed to the subscription queue, not consumed ones; `close()` discards the queue.
+`Table.subscribe` resolves audit events through `eventFromAudit` for both replay and live delivery: unless `includeSuperseded` (defaulting to `rawEvents`) is enabled, record mutations must match the current primary version; an absent primary entry cannot establish a current version and is skipped. Publishing advances that primary version too: a put followed by a publish replays the message but not the put under the default rule. Messages and control events are independent. Version filtering precedes historical reconstruction and `previousCount` acceptance, while audit cursors advance over rejected entries. Single-record replay retains its bounded history walk because it also contains independent published messages; only an accepted replay event suppresses the current snapshot, and `eventFilter` also applies to that fallback. Default replay reads the current primary entry for each mutation. Every subscriber of a key on a thread receives the same audit-record object in one notify pass, so `currentEntryForAudit` reads that entry once per audit record instead of once per subscriber (`subscriptionEntryRead.test.js`). It memoizes on the record object's identity (plus store and record id), which holds only because `readAuditEntry` decodes a fresh object per log entry and a key's subscribers are walked synchronously; the memo is cleared by a microtask after each pass, so it never outlives the delivery it serves or pins a dropped table's store. Opting into superseded full records retains audit-history reconstruction costs and retention limits, including patch reconstruction on durable MQTT live delivery. Collection `startTime` replay leaves `subscription.startTime` — the exclusive resume cursor `DurableSubscriptionsSession.saveSubscriptions` persists — never past a key with an unhandled in-table record: RocksDB gives every record of a transaction one `txnLogKey`, so the cursor moves to a key only on reaching the next key or finishing, and an early return (failed send, close at a yield or drain) leaves it up to one transaction back, which the resume re-sends (`subscriptionReplay.test.js`, "cursor after replay stops early"). The cursor counts events handed to the subscription queue, not consumed ones; `close()` discards the queue.
 
 ## Audit-store `'committed'` notification batching (`transactionBroadcast.ts`)
 
 The cross-thread subscription path (default `crossThreads`) drives every `Table.subscribe()` consumer. When the database's audit store emits `'committed'`, we walk the audit log via a reusable iterator and dispatch matching records to subscribers. These properties of this path are easy to break and worth knowing about before changing it:
 
-- **`databaseSubscriptions.activeCount`** is the count of live `Subscription` instances on a database. It is incremented at the end of `addSubscription` (after the Subscription is created, so the `scope: 'full-database'` early-return path correctly skips counting) and decremented in `Subscription.end()`. `notifyFromTransactionData` short-circuits when this is zero — the reusable rocksdb iterator stays put and resumes from its position the next time a subscriber arrives. Without this short-circuit, an idle database with no subscribers still pays the audit-log iteration cost on every commit during replication backlog catch-up.
+- **`databaseSubscriptions.activeCount`** is the count of live `Subscription` instances on a database. It is incremented at the end of `addSubscription` (after the Subscription is created, so the `scope: 'full-database'` early-return path correctly skips counting) and decremented in `Subscription.end()`. `notifyFromTransactionData` short-circuits when this is zero — the reusable rocksdb iterator, or LMDB's `lastTxnTime` cursor (seeded from `Date.now()` at the database's first subscribe), stays put and resumes from its position the next time a subscriber arrives. That subscriber's first pass can therefore carry mutations committed before it subscribed: `eventFromAudit` drops the superseded ones, and a still-current one can be delivered again. Without this short-circuit, an idle database with no subscribers still pays the audit-log iteration cost on every commit during replication backlog catch-up.
 - **`notifyScheduled` + `setImmediate`** in the `'committed'` listener defers the iteration off the commit microtask. Multiple `'committed'` events that land in the same event-loop turn collapse into one notify pass. `notifyScheduled` stays set for the entire drain — including across yield-and-resume turns — so a re-entry from a new `'committed'` event cannot spawn a second concurrent notify on the same iterator.
 - **Batched yielding** in `notifyFromTransactionData` (`NOTIFY_BATCH_SIZE`) is gated by `allowYield`. The `'committed'` path passes `allowYield = true`; the `listenToCommits` (same-thread `aftercommit`) path does not, because that path holds an inter-thread `'thread-local-writes'` lock that must not span event-loop turns. `subscribersWithTxns` is carried across yields via `subscriptions.pendingTxnSubscribers` so the `end_txn` signal fires exactly once when the iterator truly drains. When `activeCount` drops to zero mid-yield, the next continuation drops the carry-over to avoid invoking ended subscribers' listeners.
 - **The ancestor key walk must strictly shrink.** `notifyFromTransactionData` notifies a changed key and then each `/`-delimited ancestor (`a/b/c` → `a/b/` → `a/` → the `null` root). It stops when the computed parent equals the current key, because a key rooted at `/` otherwise yields `/` as its own parent forever, and this loop is synchronous inside the notify pass with the batch yield outside it, so a single non-terminating walk seizes every worker holding a subscription on the table (harper#2687). Any change to the parent computation must keep a `/`, `/foo`, or `//x` key terminating in `unitTests/resources/transactionBroadcastKeyWalk.test.js`.
 - **Ending a subscription never reshapes a subscriber array a delivery loop is walking.** Listeners run synchronously inside the fan-out and can end their own or any other subscription there: a throwing `rowFilter`/`eventFilter` closes one through `failSubscription`, and a `'data'` listener can call `end()`. Splicing in place shifted the next subscriber past the loop, which then silently missed that record (harper#2771). `Subscription.end()` therefore marks the subscription ended at once — it nulls `subscriptions`, the liveness signal `Table.subscribe` also reads, and decrements `activeCount` — but when its key's array is being walked it only sets the array's `hasEnded`. Each array counts the loops walking it in `traversals`, and the loop that brings the count back to zero compacts the ended entries out, detaching an emptied key and table as `end()` would. Outside a walk, `end()` removes the subscription immediately. Each loop walks only the array's length at its start, so a subscription added to a key during that key's delivery starts with the next record. The `reload` walk visits at most as many keys as the table had when it started (a key added during the walk can take the place of one that left). Without both bounds, a listener that ends and re-subscribes extends the loop forever. The per-key, `reload` and `end_txn` loops all skip a subscription whose `subscriptions` is null. A new loop over these arrays needs the same count, bound and check (`unitTests/resources/transactionBroadcastEndDuringDelivery.test.js`).
-
-## Audit-entry removal loops must track every `removeAuditEntry()`/`removeEntry()` promise
-
-`scheduleAuditCleanup` (`auditStore.ts`) and `Table.deleteHistory` (`Table.ts`, the LMDB path behind
-`delete_transaction_logs_before`) both iterate a range of audit records and remove each one. Any loop
-that removes audit/primary-store entries in a batch must attach a rejection handler to every removal
-immediately and drain all tracked promises before returning — never stash a per-iteration promise in
-an outer variable to await only the last one: an overwritten promise's rejection is never awaited or
-caught, and surfaces later as an unhandled rejection with no log to explain it. `Table.deleteHistory`
-allows up to 1,000 LMDB removals in flight
-(ten for RocksDB) so storage writes batch without growing an unbounded pending
-set. Live removals are tracked in a `Set`, and each one removes itself and wakes at most one parked
-producer when it settles, so any completion releases the loop. In these removal loops, do not repeatedly
-race the live set: each race attaches another reaction to every long-pending removal. Both phases drain
-their tracked removals before settling, including when iteration throws. `scheduleAuditCleanup` remains
-sequential because it is an automatic background loop, and it ends a pass at its first failed removal
-rather than continuing past it: its `deleted` count (the backoff input) and the `last-removed` marker it
-writes then cover a contiguous removed prefix — nothing above that marker was removed by the pass — and
-the next pass retries the failed entry first. Past a failure there is no key the marker can truthfully
-record, and counting failures as progress re-armed an all-failing pass at 10 ms. The guarantee is
-pass-local: the marker is written after its removals, `deleteHistory` never writes it, and a marker an
-older build persisted past a failed entry is not repaired; completeness is the audit floor's question.
-Stopping costs liveness only for an entry that fails on every pass, which is why `removeAuditEntry` must
-fail only when `auditStore.remove()` does (below). Regression: `auditLog.test.js` "ends a pass at a failed
-removal".
-
-Individual removal failures are logged and excluded from the returned count, but a purge that attempted
-at least one removal and completed none rejects with the first error after both phases have drained.
-Without that, `delete_transaction_logs_before` reports a successful `entries_deleted: 0` whether nothing
-was eligible or the store rejected every write, and an operator pruning to bound disk growth has no signal
-that pruning did nothing. Drain first, then decide: a failing store should still get every removal it can
-accept, and a single success means the purge made progress and reports normally.
-
-The optional primary-store cleanup snapshots each tombstone's key and version before yielding and passes
-that version to `remove()`. LMDB enforces the condition natively. Harper's RocksDB adapter re-reads and
-removes inside one native transaction, retrying a conflict once, because rocksdb-js's `remove()` accepts
-an options object rather than an LMDB-style version argument. Never replace this with a separate live read
-followed by an unconditional remove: a record recreated between those operations would be deleted.
-
-`removeAuditEntry` has a second, nested version of the same hazard: for a `'delete'`-type audit record it
-also invokes a per-table delete callback (`addDeleteRemovalCallback`) that removes the corresponding
-primary-store tombstone. That callback's promise must be returned and joined with the audit-store
-removal (currently via `Promise.all`, with the callback's own rejection — and a throwing tombstone
-lookup, such as lmdb-js `getEntry(undefined)` for an undecodable recordId — caught and logged through
-`warnContained`, so neither a failed tombstone cleanup nor a throwing log sink gets misreported as a failed
-audit-entry removal) — otherwise the tombstone removal is fire-and-forget and the same detached-rejection
-hazard reappears one level down.
-A tombstone whose cleanup fails this way is not swept automatically — `scheduleAuditCleanup`'s automatic
-pass never retries it, since the audit entry that would have triggered a retry is already gone. It sits
-in the primary store until an operator runs `delete_transaction_logs_before` with `cleanup_deleted_records: true`.
-
-## Audit retention cleanup is a self-rearming, engine-independent lifecycle
-
-One call to `scheduleAuditCleanup` establishes a retention cadence that ends when the root store closes
-(or immediately in process-wide read-only mode). Storage-engine selection changes the work inside each pass, not whether the timer,
-serialization barrier, error containment, and re-arm exist. LMDB removes bounded batches of audit
-entries; RocksDB asks rocksdb-js to purge conservatively eligible log segments before the same time
-cutoff. Disk-pressure callbacks may accelerate the next pass and shorten the effective window, but
-ordinary retention progress must not depend on pressure.
-
-The two engines do not share a cadence rule, because their units of progress differ. LMDB's adaptive
-backoff reads a per-entry delete count: it speeds up while entries are being removed and doubles while
-idle. Rocks reclaims whole segments whose eligibility changes only on rotation/flush, so the same
-signal would only make it rescan the same files — its delay is instead a pure function of the
-pressure-adjusted retention window (a tenth of it, floored at `DEFAULT_AUDIT_CLEANUP_DELAY`).
-
-Exactly one Rocks purge loop exists per store, and that is owned by the **arming** sites, not the
-re-arm: `onStorageReclamation` registers its handler only on the last worker (it takes no
-`skipThreadCheck`), and the store-open arm gates on the same index. The last-worker conjunct on the
-re-arm is therefore unreachable through either of those paths; it is a backstop for a direct caller
-of the exported `scheduleAuditCleanup`, because a store-wide segment purge looping on every worker is
-duplicated work. If a future change passes `skipThreadCheck: true` at the registration site, that
-backstop — not the registration — becomes the thing keeping the loop single.
-
-Both re-arm guards are **Rocks-only**. The LMDB arm re-arms unconditionally, so it neither yields to
-an already-pending pass (a pressure-armed 100ms pass can be cancelled and replaced by the idle
-backoff) nor restricts itself to one worker — pre-existing LMDB behavior, not an invariant.
-
-Two things a purge does **not** need to coordinate, both load-bearing for the continuous cadence.
-Unlinking a segment a consumer has mapped is safe **on POSIX**: the inode outlives the unlink, and the
-mapping cache (`_logBuffers`) holds `WeakRef`s, with a strong ref only on the newest segment, which is
-never purge-eligible — so nothing pins a purged inode and no cross-worker cache invalidation is
-required. Windows does not share that property: deleting a mapped segment raises a sharing violation,
-so the purge throws, is warn-logged, re-arms, and makes no progress for as long as a consumer holds the
-mapping. The continuous cadence therefore turns a Windows retention stall into a steady state rather
-than a one-off, and nothing covers it — the Rocks retention integration test skips win32.
-What is _not_ covered is the segment a lagging consumer has not mapped yet: `TransactionLog.query()`'s
-iterator returns `done` when its next segment cannot be mapped, indistinguishable from being caught up
-(rocksdb-js `src/transaction-log-reader.ts`). A consumer that far behind needs a full copy rather than
-log replay, so the gap is a missing escalation signal in the reader, not a reason to hold retention —
-tracked as HarperFast/rocksdb-js#805. Continuous retention is what moves it from unreachable-in-steady-state
-to routine: a peer offline longer than `logging.auditRetention` now resumes into a purged prefix and is
-recorded as caught up, and `txnlogReplayGapBytes` observes the gap without escalating on it.
-
-Retirement is two things, and teardown needs both. `stopAuditCleanup()` latches the loop closed and
-cancels the pending timer, and it **returns a drain barrier** — a promise that settles once the pass
-already running has finished. The barrier is what makes closing stores safe: lmdb-js stamps the DBI
-number into its write instruction synchronously and the native writer consumes it later
-(`node_modules/lmdb/write.js`), so a pass suspended inside `await removeAuditEntry()` still has a
-delete pending against the primary and audit DBIs, and LMDB forbids closing a DBI an existing
-transaction has modified. `dropDatabase()` and the legacy arm of `Table.dropTable()` await it.
-`closeDatabase()` and branch `close()` close commit admission, then drain tracked Rocks transactions,
-table maintenance, audit cleanup, and derived-index work before closing stores. Every environment
-touch remaining in a resumed cleanup pass — cursor advance, cursor release, marker write, re-arm —
-also re-checks `rootStore.status`. `resetDatabases()` closes LMDB roots with no retirement call at all,
-so that re-check is a routine
-path rather than a defensive one.
-
-The last-removed marker is retained until it commits. A rejected write is logged and carried to the
-next pass rather than dropped: a pass that deletes nothing never reaches the write again, so one
-transient failure would otherwise leave the recorded boundary permanently behind the entries that
-were already removed.
+- **Closing with a final message always finishes, then rethrows.** `IterableEventQueue.close(finalMessage)` delivers the message while `closed` is still false. Then, in a `finally`, it sets `closed`, settles a waiting iteration and emits `'close'`, so `Subscription.end()` deregisters the subscription even when a `'data'` listener throws on the message. A waiting iterator gets the final message before it completes, and `'close'` fires once even if a listener closes the queue while handling the message. The listener's exception is rethrown. If a `'close'` listener also throws, its error replaces the first. A caller that must not throw catches it: `failSubscription` in `Table.subscribe` logs it, so nothing escapes into the replay's `.catch` or the fan-out, and `endSubscriptionsFromEarlierHandles` warns and goes on to the next subscription (`IterableEventQueue.test.js`; `subscriptionReplay.test.js`, "a replay that fails"; `transactionBroadcastEndDuringDelivery.test.js`).
 
 ## `createBlob(readable)` and `table.put()` don't synchronously drain the source
 
@@ -882,6 +650,10 @@ the deprecated `delete_audit_logs_before` op _requires_ `table`, so it always er
 bridge use `!= null` presence, not truthiness, so a table named `"0"` addressed numerically stays
 table-scoped instead of widening to a database purge.
 
+## Boot replay reads bypass the primary-record cache
+
+`DatabaseTransaction.save` reads replay commit bases with `uncachedRead`: WeakRef targets survive a synchronous replay job even after cache eviction, so caching already-durable records behind a stale watermark retained the whole scanned backlog (harper#2950); enforced by `replayCommitBoundaries.test.js`'s cache-identity and crash-reopen heap-growth tests.
+
 ## A numeric transaction-log selector is a lookup, never a log name (`RocksTransactionLogStore.getRange`)
 
 `useLog` is get-or-create, so only a string log name may reach it; a node id with no log (a relayed or removed origin, whose writes `put()` routes to the via-node or `local` log) is an empty range ([harper#2778](https://github.com/HarperFast/harper/issues/2778)). Pinned by `auditLog.test.js` "a numeric log id with no log misses without creating one".
@@ -904,8 +676,8 @@ degrades to the historical behavior rather than replacing it. Invariants that ar
   execution truncates + filters (wider range than the estimable one). Two ways this has already
   been got wrong: `lt`/`le` need `searchByIndex`'s `start: true` lower bound, or the estimate
   counts the `[null, primaryKey]` entries an `indexNulls` index holds and execution skips (`true`
-  sorts above `null`) — measured at 21× inflation on an index that is 99% nulls, which is worse
-  than the flat heuristic it replaces; and `RocksIndexStore` must widen value-space bounds to
+  sorts above `null`), which on a mostly-null index is worse than the flat heuristic it
+  replaces; and `RocksIndexStore` must widen value-space bounds to
   `[value, MAXIMUM_KEY]` composite bounds, because the base implementation's byte-successor
   semantics exclude the wrong entries on composite `[value, primaryKey]` keys. `getRange` and
   `estimateCount` therefore share one `translateIndexBounds` helper rather than two copies.
@@ -994,3 +766,16 @@ environment is a use-after-free (an intermittent segfault in the lmdb unit run).
 an LMDB root once, only while `open`, skips its dbis, and closes every alias sharing it. RocksDB
 column families are independently refcounted handles, so they are still closed one by one. Enforced by
 the shared-store close cases in `unitTests/resources/databaseAliasIdentity.test.js`.
+
+## A local-only write marks both the record and its audit entry, and replay preserves it (`Table.ts` internal writes, `replayLogs.ts`)
+
+`LOCAL_ONLY` rides both persisted forms — the audit entry's `extendedType` and the stored record's
+`metadataFlags` — so a replication sender skips it by a bitmask test without decoding the value.
+Record mutations set both from `options.localOnly`. A publish applies the option only to its message
+audit entry and preserves the existing row bit, because a message does not change the row's replication
+eligibility. The static protocol verbs never carry the option; of the instance verbs only
+`publish(target, message, options)` forwards one. Crash replay re-derives it from the entry, because
+replay re-encodes the record but never re-appends its audit entry (replay transactions are `isRetry`).
+The row bit reflects its latest mutation, so a caller needing a row to stay local re-asserts it on every
+mutation. Reload and derived-index `evict` markers are always local-only; lock control entries never are.
+Enforced by `unitTests/resources/localOnly.test.js` (both engines; crash + boot replay on RocksDB).