@zakkster/lite-bake-stream 1.4.1 → 1.5.0

package/CHANGELOG.md

@@ -2,6 +2,68 @@
 
 All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.5.0] -- 2026-09-02
+
+M5 -- every ceiling gets a door; the shard budget counts all the bytes it ships; the floors are measured. BS-26 closed; BS-27's floor items closed (the writer-holds-all-shards ~2x container peak moves to M6's streaming emission); BS-32/BS-33/BS-34 closed. Suite: 461 -> 479 tests / 479 pass / 0 fail / 0 todo. Torture: 44/44 fast (~322 ms tier sum), 47/47 full (~5.6 s) incl. a 2100-shard directory-scale scenario (SPEC checker green, RangeReader LRU cap held at 8, `_maybeEvict` allocation-flat over 20000 ops); arrayBuffers growth 0.00 MB both tiers; t7 soak tracker size 0. Inventory gate: 54 -> 57 thrown codes / 0 unpinned. Falsifiability, measured item-by-item with baseline src stashed: all 15 new door/behavior assertions fail on baseline and pass on this tree, every stay-green net holds on both, and an all-F64-lane container is byte-identical to the 1.4.1 output (same sha256 and length). BREAK matrix {1, t6, l, m, j} all exit non-zero.
+
+### Added
+
+- **Serialization ceilings (BS-26).** `StringTable` now fails closed with a new
+  `StringTableError` code `ST_BLOB_OVERFLOW` when a per-shard string table would
+  cross the u32 `blob_length` (4294967295) or `entry_count` (4294967294) ceiling,
+  instead of wrapping silently on `setUint32` and minting a corrupt-but-in-bounds
+  container. The doubling paths clamp their allocation at the ceiling so every
+  crossing write re-enters the guard, making placement provably complete.
+  `StringTableError` is re-exported from `index.js` and declared in the types.
+  `PreserveWriter.onRecord` raises the same `ST_BLOB_OVERFLOW` (shared code)
+  before a shard's `payload_len` (blob + trailing offset table) would exceed u32.
+  A test-only `__setStringTableLimits({blobBytes, entryCount})` seam (no semver
+  guarantee, absent from the public API/types/docs) lowers the ceilings for a
+  cheap gated crossing on the real grow path.
+- **Row-index doors (BS-32).** `Reader.get`, `MultiReader.get`, `RangeReader.get`
+  and `PreserveReader.getBytes/getString/getJSON` now reject a negative,
+  fractional, `NaN`, or out-of-range `rowIdx` with `R_ROW_OUT_OF_RANGE`
+  (`M_ROW_OUT_OF_RANGE` for `MultiReader`) rather than returning `undefined` or a
+  garbage subarray from a fractional byte offset. `Reader.get` uses a `>>>0`
+  compare; the others use `Number.isInteger` (they legitimately address up to
+  2^53). The SHARD-index escape hatches keep their unchecked raw-access contract.
+- **Safe-integer offset guard (BS-26).** `Reader`, `RangeReader` and
+  `PreserveReader` narrow every u64 header/schema/directory offset through a cold
+  `u64()` helper that throws `R_OFFSET_TOO_LARGE` past 2^53-1, where a `Number()`
+  cast would lose precision and hand a corrupted offset to every bounds check.
+- **`RangeReader` adapter validation (BS-34 + BS-33).** All seven `adapter.fetch` sites
+  funnel through `_fetchExact`, which rejects a non-`Uint8Array` or wrong-length
+  return with `R_ADAPTER_SHORT_READ`. `syncRange` over a not-prefetched shard now
+  throws its own `R_NOT_PREFETCHED`, distinct from the `R_TRUNCATED` truncation
+  door.
+
+### Changed
+
+- **Shard-byte budget (D2).** The schema `Writer` now rolls a shard when its
+  payload plus string-table bytes reach `targetShardBytes`, not on the row
+  ceiling alone, so a string-heavy shard no longer overshoots the target. The
+  string-table byte length is tracked incrementally with zero allocation (one
+  compare per string value, updated only on a unique arrival) and is exact versus
+  `StringTable.serialize` at every finalize. The budget reads only
+  chunk-invariant state, so re-chunked input still yields byte-identical
+  containers (the t0 law), and a container whose schema has only F64 lanes keeps
+  a zero string budget and is **byte-identical** to the pre-D2 output.
+- **Floors.** The schema `Writer` field-name lookup moves from an O(F) hash-scan
+  to an O(1) `Map` (byte-confirmed, so a real FNV collision such as
+  `gwzx`/`16cd` still resolves correctly): on an ad-hoc 64-/256-field
+  flat-record serialize microbench (`bench/bench-tokenizer.js` carries a fixed
+  7-field fixture, so it cannot express this floor), throughput went from
+  190.0 -> 216.7 MB/s at 64 fields and 113.3 -> 213.6 MB/s at 256 fields --
+  the O(F) per-key term no longer degrades with field count. qa's independent
+  microbench (different corpus) corroborates the shape: baseline fell
+  128.0 -> 77.3 MB/s going 64 -> 256 fields; with the map it holds
+  163.0 -> 167.9 MB/s. The sample-window key
+  decoder is now a shared module-scope `TextDecoder` instead of one per key.
+  `PreserveWriter` reuses its working buffers across shard rolls (they were
+  reallocated every roll) -- the reuse is byte-safe (every read is bounded by the
+  reset cursors) and holds the working store byte-flat under a steady-stream soak.
+- `package.json` exports: all 12 entries (root + 11 subpaths) gain the `node` condition alongside `types`/`import`/`default`, matching the suite convention. Verified: 12/12 subpaths resolve via self-reference import, `require()` resolves through `node` on Node 26 (`require(esm)`), npm test 461/461, `npm pack --dry-run` unchanged at 32 files.
+
 ## [1.4.1] -- 2026-09-02
 
 M8 -- the four reserved torture cells become real; BS-30 and BS-31 close. No tier prints "reserved" any more. Suite: 461 tests / 461 pass / 0 fail / 0 todo (unchanged -- no named-suite edits). Torture: 44/44 fast (~1.3 s wall), 47/47 full, arrayBuffers growth 0. BREAK matrix: ten runs (`=1`, eight registry ids, one unknown id) all exit non-zero; the unknown id is refused at import. Inventory gate: 54 thrown codes / 0 unpinned. Falsifiability: with baseline src stashed, exactly one new assertion fails (the onProgress-identity pin), both tiers. Test-only except one hot-path reuse in `src/FileIngest.js` (below).

package/README.md

@@ -13,7 +13,7 @@
 
 > Streaming byte-level JSON compiler for [`@zakkster/lite-bake`](https://github.com/PeshoVurtoleta/lite-bake). Zero-GC, tree-shakeable, gigabyte-scale.
 
-**Status:** v1.4.1. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak with zero GC and byte-exact preservation across 590 million cells.
+**Status:** v1.5.0. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak with zero GC and byte-exact preservation across 590 million cells.
 
 ## Two modes, one API
 

package/llms.txt

@@ -8,13 +8,13 @@ Ingest gigabyte-scale JSON (top-level array or NDJSON) into the `lite-bake` LBK1
 
 ## Status
 
-v1.4.1 — stable. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak (M1 MacBook Pro): 98.37M rows, 4.89 GB container, zero major GC, zero minor GC, 499 KB total heap allocation, 590.21M cells verified byte-exact, zero mismatches. Tokenizer benches at 222-237 MB/s (~55% of JSON.parse, with no object graph allocated).
+v1.5.0 — stable. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak (M1 MacBook Pro): 98.37M rows, 4.89 GB container, zero major GC, zero minor GC, 499 KB total heap allocation, 590.21M cells verified byte-exact, zero mismatches. Tokenizer benches at 222-237 MB/s (~55% of JSON.parse, with no object graph allocated).
 
 Public API follows semver from 1.0.0. Future additions (I64 lane, columnar payload mode, container-level string table) land via the format's forward-compat seams -- `min_reader_version` on ShardEntry, reserved FieldDescriptor flags, the `metadata_off` block wrapper -- without a format_version bump.
 
 See SPEC.md for the LBK1 container format, section 3.6 for zone maps, section 4.3 for the reserved field flags.
 
-## Public API (v1.4.1)
+## Public API (v1.5.0)
 
 Two ingest modes share one top-level API:
 
@@ -62,6 +62,17 @@ Error classes with stable `code`: `TokenizerError`, `WriterError`, `ReaderError`
 
 Every declared field of every row round-trips. F64 lanes: bit-exact for numeric literals with ≤15 significant digits and |exponent| ≤ 22 (Clinger's fast path); ≤1 ULP drift outside that domain (documented, pinned by NumericBoundary.test.js). U32 lanes: byte-exact UTF-8, unconditional. Missing fields → documented defaults. Unknown keys → silently dropped. Wrong-type value on post-freeze schema → W_LANE_MISMATCH error, no corrupt container produced. See SPEC section 7 for the full table. Asserted by tests, property-based fuzz, AND torture-scale verification (~73,500 rows per fast-tier run).
 
+## Row-index and refusal codes
+
+- Random-access row surfaces fail closed on a bad `rowIdx` (negative, fractional, NaN, or >= totalRows): `Reader.get`, `RangeReader.get` and `PreserveReader.getBytes/getString/getJSON` throw `R_ROW_OUT_OF_RANGE`; `MultiReader.get` throws `M_ROW_OUT_OF_RANGE`. The SHARD-index escape hatches (`shardPayload`, `shardF64`, `shardStringTable`, `loadShard`) keep their unchecked raw-access contract; `shardBounds` returns null out-of-range.
+- `R_OFFSET_TOO_LARGE`: a u64 header/schema/directory offset exceeds 2^53-1 (a Number() cast would lose precision). Raised by all three reader classes.
+- `R_NOT_PREFETCHED`: `RangeReader.syncRange` over a shard `prefetchRange` has not cached (distinct from the `R_TRUNCATED` truncation door). `R_ADAPTER_SHORT_READ` also covers a non-Uint8Array or wrong-length adapter return.
+- `ST_BLOB_OVERFLOW`: a per-shard string table would cross the u32 blob_length (4294967295) or entry_count (4294967294) ceiling; also raised by `PreserveWriter` before a shard payload_len would exceed u32. One shared code across `StringTable` and `PreserveWriter`. `StringTableError` is exported from the root and the `string-table` subpath.
+
+## Shard budget
+
+Schema-mode shards roll when payload bytes plus string-table bytes reach `targetShardBytes`, not on the row ceiling alone. The string-table byte count is tracked with zero allocation and is chunk-invariant, so re-chunked input yields byte-identical containers and an all-F64 schema is byte-identical to the pre-budget output. `PreserveWriter` reuses its working buffers across shard rolls.
+
 ## Numbers
 
 F64 only. Values exceeding IEEE 754 double range are rejected as `E_NUMBER_OVERFLOW`. 64-bit integer IDs above 2^53 lose precision silently (documented in SPEC 5.4); v2 will introduce an opt-in bytes-preserved lane.

package/package.json

@@ -1,67 +1,79 @@
 {
   "name": "@zakkster/lite-bake-stream",
-  "version": "1.4.1",
+  "version": "1.5.0",
   "description": "Streaming byte-level JSON to lite-bake binary compiler. Zero-GC, tree-shakeable, gigabyte-scale.",
   "type": "module",
   "sideEffects": false,
   "exports": {
     ".": {
       "types": "./types/index.d.ts",
+      "node": "./src/index.js",
       "import": "./src/index.js",
       "default": "./src/index.js"
     },
     "./tokenizer": {
       "types": "./types/Tokenizer.d.ts",
+      "node": "./src/Tokenizer.js",
       "import": "./src/Tokenizer.js",
       "default": "./src/Tokenizer.js"
     },
     "./writer": {
       "types": "./types/Writer.d.ts",
+      "node": "./src/Writer.js",
       "import": "./src/Writer.js",
       "default": "./src/Writer.js"
     },
     "./reader": {
       "types": "./types/Reader.d.ts",
+      "node": "./src/Reader.js",
       "import": "./src/Reader.js",
       "default": "./src/Reader.js"
     },
     "./string-table": {
       "types": "./types/StringTable.d.ts",
+      "node": "./src/StringTable.js",
       "import": "./src/StringTable.js",
       "default": "./src/StringTable.js"
     },
     "./file-ingest": {
       "types": "./types/FileIngest.d.ts",
+      "node": "./src/FileIngest.js",
       "import": "./src/FileIngest.js",
       "default": "./src/FileIngest.js"
     },
     "./range-reader": {
       "types": "./types/RangeReader.d.ts",
+      "node": "./src/RangeReader.js",
       "import": "./src/RangeReader.js",
       "default": "./src/RangeReader.js"
     },
     "./multi-reader": {
       "types": "./types/MultiReader.d.ts",
+      "node": "./src/MultiReader.js",
       "import": "./src/MultiReader.js",
       "default": "./src/MultiReader.js"
     },
     "./split": {
       "types": "./types/Split.d.ts",
+      "node": "./src/Split.js",
       "import": "./src/Split.js",
       "default": "./src/Split.js"
     },
     "./preserve-tokenizer": {
       "types": "./types/PreserveTokenizer.d.ts",
+      "node": "./src/PreserveTokenizer.js",
       "import": "./src/PreserveTokenizer.js",
       "default": "./src/PreserveTokenizer.js"
     },
     "./preserve-writer": {
       "types": "./types/PreserveWriter.d.ts",
+      "node": "./src/PreserveWriter.js",
       "import": "./src/PreserveWriter.js",
       "default": "./src/PreserveWriter.js"
     },
     "./preserve-reader": {
       "types": "./types/PreserveReader.d.ts",
+      "node": "./src/PreserveReader.js",
       "import": "./src/PreserveReader.js",
       "default": "./src/PreserveReader.js"
     }

package/src/FileIngest.js

@@ -27,7 +27,7 @@ import { PreserveWriter } from './PreserveWriter.js';
 import { PreserveReader } from './PreserveReader.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 const U32_MAX = 4294967295;
 const INGEST_OPTS = {

package/src/MultiReader.js

@@ -25,7 +25,7 @@
 //   M_ROW_OUT_OF_RANGE    - rowIdx >= totalRows
 //   M_TOO_MANY_ROWS       - cumulative row count exceeds Number.MAX_SAFE_INTEGER
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 export class MultiReaderError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'MultiReaderError'; }
@@ -100,7 +100,10 @@ export class MultiReader {
 
   // Which reader contains global row rowIdx? Returns { readerIdx, localRow }.
   _locateRow(rowIdx) {
-    if (rowIdx < 0 || rowIdx >= this._totalRows) {
+    // BS-32: Number.isInteger (not >>>0) because MultiReader legitimately
+    // addresses up to 2^53 (see M_TOO_MANY_ROWS). Rejects fractional, NaN, and
+    // out-of-range in one guard; covers get() and readerForRow().
+    if (!Number.isInteger(rowIdx) || rowIdx < 0 || rowIdx >= this._totalRows) {
       throw new MultiReaderError('M_ROW_OUT_OF_RANGE',
         'rowIdx ' + rowIdx + ' out of range [0, ' + this._totalRows + ')');
     }

package/src/PreserveReader.js

@@ -28,11 +28,12 @@
 //   R_BAD_FOOTER             - footer magic_end or footer_len is malformed
 //   R_INVALID                - a structure is internally inconsistent but in-bounds
 //   R_SHARD_VERSION_TOO_NEW  - a shard's min_reader_version > this reader
-//   R_ROW_OUT_OF_RANGE       - rowIdx >= totalRows
+//   R_ROW_OUT_OF_RANGE       - rowIdx negative, fractional, NaN, or >= totalRows (BS-32)
+//   R_OFFSET_TOO_LARGE       - a u64 header/directory offset exceeds 2^53-1
 
 import { toContainerBuffer } from './Views.js';
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 const CONTAINER_HEADER_BYTES = 48;
 const SHARD_ENTRY_BYTES = 40;
@@ -42,6 +43,17 @@ export class PreserveReaderError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'PreserveReaderError'; }
 }
 
+// Read a u64 header/directory field and narrow it to a JS number, failing closed
+// past Number.MAX_SAFE_INTEGER. Past 2^53-1 a Number() cast loses precision and
+// every downstream bounds check reads a corrupted offset (BS-05). Cold: parse-only.
+function u64(dv, off, what) {
+  const v = dv.getBigUint64(off, true);
+  if (v > 9007199254740991n)
+    throw new PreserveReaderError('R_OFFSET_TOO_LARGE',
+      what + ' value ' + v + ' exceeds the safe-integer ceiling 9007199254740991');
+  return Number(v);
+}
+
 export class PreserveReader {
   static fromBuffer(input) {
     return new PreserveReader(toContainerBuffer(input, 'PreserveReader.fromBuffer'));
@@ -84,11 +96,11 @@ export class PreserveReader {
     if (reserved1 !== 0)
       throw new PreserveReaderError('R_RESERVED_NONZERO', 'header reserved1 at offset 36 must be 0, got ' + reserved1);
 
-    this._schemaBlockOff = Number(this._dv.getBigUint64(8, true));
-    this._metadataOff    = Number(this._dv.getBigUint64(16, true));
-    this._shardDirOff    = Number(this._dv.getBigUint64(24, true));
+    this._schemaBlockOff = u64(this._dv, 8, 'schema_block_off');
+    this._metadataOff    = u64(this._dv, 16, 'metadata_off');
+    this._shardDirOff    = u64(this._dv, 24, 'shard_directory_off');
     this._shardCount     = this._dv.getUint32(32, true);
-    this._totalRows      = Number(this._dv.getBigUint64(40, true));
+    this._totalRows      = u64(this._dv, 40, 'total_rows');
 
     if (this._schemaBlockOff !== 0) {
       throw new PreserveReaderError('R_INVALID', 'preserve container has non-zero schema_block_off');
@@ -127,14 +139,14 @@ export class PreserveReader {
     let cumulativeRow = 0;
     for (let i = 0; i < this._shardCount; i++) {
       const entryOff = this._shardDirOff + i * SHARD_ENTRY_BYTES;
-      const payloadOff = Number(this._dv.getBigUint64(entryOff + 0, true));
+      const payloadOff = u64(this._dv, entryOff + 0, 'shard ' + i + ' payload_off');
       const payloadLen = this._dv.getUint32(entryOff + 8, true);
       const rowCount   = this._dv.getUint32(entryOff + 12, true);
       const minReaderVer = this._dv.getUint16(entryOff + 16, true);
       const shardFlags   = this._dv.getUint16(entryOff + 18, true);
       const shardReserved = this._dv.getUint32(entryOff + 20, true);
-      const localStrOff  = Number(this._dv.getBigUint64(entryOff + 24, true));
-      const localStrLen  = Number(this._dv.getBigUint64(entryOff + 32, true));
+      const localStrOff  = u64(this._dv, entryOff + 24, 'shard ' + i + ' local_string_off');
+      const localStrLen  = u64(this._dv, entryOff + 32, 'shard ' + i + ' local_string_len');
       if (minReaderVer > 1) {
         throw new PreserveReaderError('R_SHARD_VERSION_TOO_NEW',
           'shard ' + i + ' requires reader version ' + minReaderVer);
@@ -195,7 +207,10 @@ export class PreserveReader {
   get hasZoneMaps() { return false; }
 
   _locateShard(rowIdx) {
-    if (rowIdx < 0 || rowIdx >= this._totalRows) {
+    // BS-32: Number.isInteger rejects fractional/NaN too, so getBytes(1.5) fails
+    // closed instead of returning a garbage subarray. One site covers
+    // getBytes/getString/getJSON.
+    if (!Number.isInteger(rowIdx) || rowIdx < 0 || rowIdx >= this._totalRows) {
       throw new PreserveReaderError('R_ROW_OUT_OF_RANGE',
         'rowIdx ' + rowIdx + ' out of range [0, ' + this._totalRows + ')');
     }

package/src/PreserveTokenizer.js

@@ -32,7 +32,7 @@
 
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 const U32_MAX = 4294967295;
 const PRESERVE_TOKENIZER_OPTS = {

package/src/PreserveWriter.js

@@ -18,7 +18,7 @@
 
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 export class PreserveWriterError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'PreserveWriterError'; }
@@ -48,13 +48,19 @@ export class PreserveWriter {
     this._shards = [];
     this._totalRows = 0;
     this._finalized = false;
+    this._currentBlobs = null;   // null => never allocated; _allocateShard allocates once
     this._allocateShard();
   }
 
   _allocateShard() {
-    // Pre-allocate up to targetShardBytes for zero-GC record writes.
-    this._currentBlobs = new Uint8Array(this._targetShardBytes);
-    this._currentOffsets = new Uint32Array(INITIAL_OFFSETS_CAP);
+    // Allocate the working buffers ONCE (BS-27 floor). Every subsequent shard
+    // reuses them: _finalizeCurrentShard resets the cursors instead of nulling
+    // the buffers, so a shard roll allocates nothing. A grown buffer (oversized
+    // record) simply carries forward as a larger reusable buffer.
+    if (this._currentBlobs === null) {
+      this._currentBlobs = new Uint8Array(this._targetShardBytes);
+      this._currentOffsets = new Uint32Array(INITIAL_OFFSETS_CAP);
+    }
     this._currentBlobBytes = 0;
     this._currentRowCount = 0;
   }
@@ -74,7 +80,19 @@ export class PreserveWriter {
       this._allocateShard();
     }
     // Grow shard buffer if this single record exceeds the pre-allocated size.
+    // Fail closed (BS-31) on a BLOB-dimension crossing: this guard covers a
+    // record whose blob bytes push payload_len past the u32 ceiling (including
+    // the oversized-single-record path), and the _growBlobBuffer clamp keeps the
+    // buffer <= U32_MAX so such a write always re-enters this grow branch. The
+    // ORTHOGONAL crossing -- payload_len exceeding u32 via OFFSET-TABLE
+    // accumulation (many tiny records, blob under target, grow branch never
+    // firing) -- is caught in _finalizeCurrentShard. ST_BLOB_OVERFLOW is the
+    // shared cross-writer overflow code (R11).
     if (this._currentBlobBytes + len > this._currentBlobs.length) {
+      if (this._currentBlobBytes + len + (this._currentRowCount + 1) * 4 > U32_MAX)
+        throw new PreserveWriterError('ST_BLOB_OVERFLOW',
+          'preserve shard payload ' + (this._currentBlobBytes + len) +
+          ' bytes + offset table would exceed the u32 payload_len ceiling ' + U32_MAX);
       this._growBlobBuffer(this._currentBlobBytes + len);
     }
     // Grow offsets array if we've hit the pre-allocated ceiling.
@@ -107,6 +125,9 @@ export class PreserveWriter {
   _growBlobBuffer(needed) {
     let cap = this._currentBlobs.length;
     while (cap < needed) cap *= 2;
+    // Clamp so the buffer never exceeds the u32 payload ceiling; this makes the
+    // onRecord grow-branch guard provably complete (see onRecord).
+    if (cap > U32_MAX) cap = U32_MAX;
     const next = new Uint8Array(cap);
     next.set(this._currentBlobs.subarray(0, this._currentBlobBytes));
     this._currentBlobs = next;
@@ -124,6 +145,15 @@ export class PreserveWriter {
     const blobLen = this._currentBlobBytes;
     const offsetTableBytes = rowCount * 4;
     const shardBytesLen = blobLen + offsetTableBytes;
+    // Fail closed (BS-31, cold) on an OFFSET-TABLE-driven crossing: payload_len =
+    // blob bytes + rowCount*4 is written into the shard directory as a u32, and
+    // enough tiny records can push it past the ceiling without the onRecord grow
+    // branch ever firing. This is the door that branch cannot cover. Shared code
+    // ST_BLOB_OVERFLOW (R11).
+    if (shardBytesLen > U32_MAX)
+      throw new PreserveWriterError('ST_BLOB_OVERFLOW',
+        'preserve shard payload_len ' + shardBytesLen + ' (blob ' + blobLen + ' + offset table ' +
+        offsetTableBytes + ') exceeds the u32 ceiling ' + U32_MAX);
 
     const shardBytes = new Uint8Array(shardBytesLen);
     shardBytes.set(this._currentBlobs.subarray(0, blobLen), 0);
@@ -137,10 +167,10 @@ export class PreserveWriter {
       rowCount,
       blobLen,
     });
-    // Zero out the working buffers so a stale ref can't leak old bytes into
-    // the next shard by accident. The buffers are reused via _allocateShard.
-    this._currentBlobs = null;
-    this._currentOffsets = null;
+    // Reuse the working buffers for the next shard: reset the cursors only, keep
+    // the allocations (BS-27). No stale bytes can leak -- every read into these
+    // buffers is bounded by _currentBlobBytes / _currentRowCount (both zeroed
+    // here), and the shard copy above took only subarray(0, blobLen).
     this._currentBlobBytes = 0;
     this._currentRowCount = 0;
   }

package/src/RangeReader.js

@@ -30,8 +30,11 @@
 // Error codes (stable, share prefix with Reader.js where semantics overlap):
 //   R_BAD_MAGIC / R_UNSUPPORTED_VERSION / R_UNSUPPORTED_ENDIAN / R_BAD_FIELD_FLAGS
 //   R_UNSUPPORTED_LANE / R_SHARD_VERSION_TOO_NEW / R_TRUNCATED / R_UNKNOWN_FIELD
-//   R_ADAPTER_SHORT_READ   -- adapter returned fewer bytes than requested
-//   R_ROW_OUT_OF_RANGE     -- rowIdx >= totalRows
+//   R_ADAPTER_SHORT_READ   -- adapter returned fewer bytes than requested, OR a
+//                             non-Uint8Array / wrong-length adapter return (T7c)
+//   R_ROW_OUT_OF_RANGE     -- rowIdx negative, fractional, NaN, or >= totalRows (BS-32)
+//   R_OFFSET_TOO_LARGE     -- a u64 header/schema/directory offset exceeds 2^53-1
+//   R_NOT_PREFETCHED       -- syncRange over a shard that prefetchRange has not cached
 //   R_WRONG_MODE           -- preserve-mode container fed to the schema RangeReader
 //   R_BAD_FLAGS            -- unknown header flag bits set
 //   R_RESERVED_NONZERO     -- a reserved header/descriptor/shard field is non-zero
@@ -42,7 +45,7 @@
 import { StringTable } from './StringTable.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 const CONTAINER_HEADER_BYTES = 48;
 const SHARD_ENTRY_BYTES = 40;
@@ -65,6 +68,18 @@ export class RangeReaderError extends Error {
 }
 function raiseRange(code, msg) { throw new RangeReaderError(code, msg); }
 
+// Read a u64 field from `dv` (the header, schema, or directory view -- three
+// different DataViews here) and narrow it to a JS number, failing closed past
+// Number.MAX_SAFE_INTEGER. Past 2^53-1 a Number() cast loses precision and every
+// downstream bounds check reads a corrupted offset (BS-05). Cold: parse-only.
+function u64(dv, off, what) {
+  const v = dv.getBigUint64(off, true);
+  if (v > 9007199254740991n)
+    throw new RangeReaderError('R_OFFSET_TOO_LARGE',
+      what + ' value ' + v + ' exceeds the safe-integer ceiling 9007199254740991');
+  return Number(v);
+}
+
 // Validate a local string table's shape before StringTable.parse casts a
 // Uint32Array over it (T-1..T-4). `bytes` is the already-fetched shard slice;
 // `off` and `len` locate the table within it.
@@ -205,7 +220,7 @@ export class RangeReader {
     }
 
     // Step 1: header (48 bytes) tells us schema + shard-dir offsets.
-    const headerBytes = await this.adapter.fetch(0, CONTAINER_HEADER_BYTES);
+    const headerBytes = await this._fetchExact(0, CONTAINER_HEADER_BYTES);
     const hdrDv = new DataView(headerBytes.buffer, headerBytes.byteOffset, headerBytes.byteLength);
     if (headerBytes[0] !== 0x4C || headerBytes[1] !== 0x42 || headerBytes[2] !== 0x4B || headerBytes[3] !== 0x31) {
       throw new RangeReaderError('R_BAD_MAGIC', 'header magic is not LBK1');
@@ -226,11 +241,11 @@ export class RangeReader {
     if (reserved1 !== 0)
       throw new RangeReaderError('R_RESERVED_NONZERO', 'header reserved1 at offset 36 must be 0, got ' + reserved1);
 
-    this._schemaBlockOff = Number(hdrDv.getBigUint64(8, true));
-    this._metadataOff    = Number(hdrDv.getBigUint64(16, true));
-    this._shardDirOff    = Number(hdrDv.getBigUint64(24, true));
+    this._schemaBlockOff = u64(hdrDv, 8, 'schema_block_off');
+    this._metadataOff    = u64(hdrDv, 16, 'metadata_off');
+    this._shardDirOff    = u64(hdrDv, 24, 'shard_directory_off');
     this._shardCount     = hdrDv.getUint32(32, true);
-    this._totalRows      = Number(hdrDv.getBigUint64(40, true));
+    this._totalRows      = u64(hdrDv, 40, 'total_rows');
 
     const size = this.adapter.size;
     if (this._schemaBlockOff < CONTAINER_HEADER_BYTES || this._schemaBlockOff >= size)
@@ -252,11 +267,11 @@ export class RangeReader {
     if (this._shardDirOff <= this._schemaBlockOff)
       throw new RangeReaderError('R_TRUNCATED', 'shard_directory_off ' + this._shardDirOff + ' not after schema_block_off ' + this._schemaBlockOff);
     const schemaBlockLen = this._shardDirOff - this._schemaBlockOff;
-    const schemaBytes = await this.adapter.fetch(this._schemaBlockOff, schemaBlockLen);
+    const schemaBytes = await this._fetchExact(this._schemaBlockOff, schemaBlockLen);
     this._parseSchema(schemaBytes);
 
     // Step 3: shard directory (fixed size = shardCount * 40).
-    const dirBytes = await this.adapter.fetch(this._shardDirOff, this._shardCount * SHARD_ENTRY_BYTES);
+    const dirBytes = await this._fetchExact(this._shardDirOff, this._shardCount * SHARD_ENTRY_BYTES);
     this._parseShardDirectory(dirBytes);
 
     // Step 4 (M7): zone maps. Fetched ONCE at open, cached. Enables query
@@ -274,7 +289,7 @@ export class RangeReader {
 
   async _loadFooter() {
     const size = this.adapter.size;
-    const footer = await this.adapter.fetch(size - CONTAINER_FOOTER_BYTES, CONTAINER_FOOTER_BYTES);
+    const footer = await this._fetchExact(size - CONTAINER_FOOTER_BYTES, CONTAINER_FOOTER_BYTES);
     if (footer[8] !== 0x31 || footer[9] !== 0x4B || footer[10] !== 0x42 || footer[11] !== 0x4C)
       throw new RangeReaderError('R_BAD_FOOTER', 'footer magic_end is not 1KBL');
     const fDv = new DataView(footer.buffer, footer.byteOffset, footer.byteLength);
@@ -291,7 +306,7 @@ export class RangeReader {
     // corruption (D1), not absence.
     if (this._metadataOff + 16 > this.adapter.size)
       throw new RangeReaderError('R_BAD_METADATA', 'zone maps segment header runs past the container');
-    const hdr = await this.adapter.fetch(this._metadataOff, 16);
+    const hdr = await this._fetchExact(this._metadataOff, 16);
     if (hdr[0] !== 0x30 || hdr[1] !== 0x5A || hdr[2] !== 0x4D || hdr[3] !== 0x31)
       throw new RangeReaderError('R_BAD_METADATA', 'zone maps magic is not ZM01');
     const hdrDv = new DataView(hdr.buffer, hdr.byteOffset, hdr.byteLength);
@@ -309,7 +324,7 @@ export class RangeReader {
     const restLen = fieldTableLen + fieldTablePad + shardCount * T * 8 * 2;
     if (this._metadataOff + 16 + restLen > this.adapter.size)
       throw new RangeReaderError('R_BAD_METADATA', 'zone maps segment runs past the container');
-    const rest = await this.adapter.fetch(this._metadataOff + 16, restLen);
+    const rest = await this._fetchExact(this._metadataOff + 16, restLen);
     const restDv = new DataView(rest.buffer, rest.byteOffset, rest.byteLength);
     const tracked = new Array(T);
     const fieldToPos = new Map();
@@ -357,7 +372,10 @@ export class RangeReader {
       const laneKind    = bytes[off + 4];
       const flags       = bytes[off + 5];
       const reserved2   = dv.getUint16(off + 6, true);
-      const nameStrOff  = Number(dv.getBigUint64(off + 8, true));
+      const nameStrOff  = u64(dv, off + 8, 'field ' + i + ' name_str_off');
+      // reserved3 is BigInt-compared vs 0n below, never Number()-cast, so it
+      // needs no narrowing guard -- the reserved-must-be-zero check catches any
+      // out-of-range value (BS-05).
       const reserved3   = dv.getBigUint64(off + 16, true);
       if (flags !== 0) throw new RangeReaderError('R_BAD_FIELD_FLAGS', 'field ' + i + ' has non-zero flags');
       if (reserved2 !== 0) throw new RangeReaderError('R_RESERVED_NONZERO', 'field ' + i + ' reserved2 must be 0, got ' + reserved2);
@@ -383,14 +401,14 @@ export class RangeReader {
     let cumulativeRow = 0;
     for (let i = 0; i < this._shardCount; i++) {
       const off = i * SHARD_ENTRY_BYTES;
-      const payloadOff   = Number(dv.getBigUint64(off + 0, true));
+      const payloadOff   = u64(dv, off + 0, 'shard ' + i + ' payload_off');
       const payloadLen   = dv.getUint32(off + 8, true);
       const rowCount     = dv.getUint32(off + 12, true);
       const minReaderVer = dv.getUint16(off + 16, true);
       const shardFlags   = dv.getUint16(off + 18, true);
       const shardReserved = dv.getUint32(off + 20, true);
-      const localStrOff  = Number(dv.getBigUint64(off + 24, true));
-      const localStrLen  = Number(dv.getBigUint64(off + 32, true));
+      const localStrOff  = u64(dv, off + 24, 'shard ' + i + ' local_string_off');
+      const localStrLen  = u64(dv, off + 32, 'shard ' + i + ' local_string_len');
       if (minReaderVer > READER_VERSION)
         throw new RangeReaderError('R_SHARD_VERSION_TOO_NEW',
           'shard ' + i + ' requires reader version ' + minReaderVer);
@@ -433,7 +451,10 @@ export class RangeReader {
 
   // Locate the shard containing rowIdx via binary search.
   _findShardIndex(rowIdx) {
-    if (rowIdx < 0 || rowIdx >= this._totalRows) {
+    // BS-32: Number.isInteger rejects fractional/NaN as well as out-of-range in
+    // one guard. This ONE site covers get(), prefetchRange, syncRange's bounds
+    // walk and the syncRange view's inner get.
+    if (!Number.isInteger(rowIdx) || rowIdx < 0 || rowIdx >= this._totalRows) {
       throw new RangeReaderError('R_ROW_OUT_OF_RANGE',
         'rowIdx ' + rowIdx + ' out of range [0, ' + this._totalRows + ')');
     }
@@ -447,6 +468,21 @@ export class RangeReader {
     return -1;   // unreachable given the range check above
   }
 
+  // Every adapter read funnels through here (T7c). The IOAdapter contract says
+  // fetch() MUST return exactly byteLength bytes; a short, over-long, or
+  // non-Uint8Array return would otherwise seed a DataView over the wrong extent
+  // and decode silent garbage. Fail closed with R_ADAPTER_SHORT_READ instead.
+  async _fetchExact(byteOffset, byteLength) {
+    const buf = await this.adapter.fetch(byteOffset, byteLength);
+    if (!(buf instanceof Uint8Array) || buf.byteLength !== byteLength) {
+      throw new RangeReaderError('R_ADAPTER_SHORT_READ',
+        'adapter returned ' +
+        (buf instanceof Uint8Array ? buf.byteLength + ' bytes' : Object.prototype.toString.call(buf)) +
+        ' for a ' + byteLength + '-byte request at offset ' + byteOffset);
+    }
+    return buf;
+  }
+
   // Load a shard's payload + local string table with ONE range request.
   // They are contiguous in the container by SPEC 3.4, so a single fetch covers
   // both. Returns the cached shard record.
@@ -464,7 +500,7 @@ export class RangeReader {
       throw new RangeReaderError('R_TRUNCATED',
         'shard ' + shardIdx + ' string table not contiguous with payload');
     }
-    const combined = await this.adapter.fetch(combinedOff, combinedLen);
+    const combined = await this._fetchExact(combinedOff, combinedLen);
     const payloadBytes = combined.subarray(0, s.payloadLen);
     // DataView over the payload's underlying ArrayBuffer window.
     const payloadDv = new DataView(payloadBytes.buffer, payloadBytes.byteOffset, payloadBytes.byteLength);
@@ -532,7 +568,7 @@ export class RangeReader {
     const lastShard  = this._findShardIndex(lastRow - 1);
     for (let s = firstShard; s <= lastShard; s++) {
       if (!this._shardCache.has(s)) {
-        throw new RangeReaderError('R_TRUNCATED',
+        throw new RangeReaderError('R_NOT_PREFETCHED',
           'syncRange requires shard ' + s + ' to be prefetched (call prefetchRange first)');
       }
     }

package/src/Reader.js

@@ -21,11 +21,20 @@
 //   R_BAD_METADATA           - metadata_off is non-zero but the zone-map segment is unparseable
 //   R_INVALID                - a structure is internally inconsistent but in-bounds
 //   R_UNKNOWN_FIELD          - get()/findShards()/shardBounds() called with an unknown field name
+//   R_OFFSET_TOO_LARGE       - a u64 header/schema/directory offset exceeds 2^53-1
+//                              (Number() would lose precision; fail closed, BS-05)
+//   R_ROW_OUT_OF_RANGE       - get(rowIdx) with rowIdx negative, fractional, NaN,
+//                              or >= totalRows (BS-32)
+//
+// Row-index policy (BS-32): get() range-checks rowIdx; the SHARD-index escape
+// hatches (shardPayload, shardF64, shardStringTable) do NOT -- their contract is
+// raw indexed access with no per-call checking. shardBounds returns null for an
+// out-of-range shard by decided policy.
 
 import { StringTable } from './StringTable.js';
 import { toContainerBuffer } from './Views.js';
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 const CONTAINER_HEADER_BYTES = 48;
 const SHARD_ENTRY_BYTES = 40;
@@ -42,6 +51,19 @@ export class ReaderError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'ReaderError'; }
 }
 
+// Read a u64 header/directory/schema field and narrow it to a JS number, failing
+// closed if it exceeds Number.MAX_SAFE_INTEGER (2^53-1). Past that a Number()
+// cast loses precision and every downstream bounds check reads a corrupted
+// offset -- an in-bounds lie is worse than a loud refusal (BS-05). Cold: called
+// only during header/schema/directory parse, never per row.
+function u64(dv, off, what) {
+  const v = dv.getBigUint64(off, true);
+  if (v > 9007199254740991n)
+    throw new ReaderError('R_OFFSET_TOO_LARGE',
+      what + ' value ' + v + ' exceeds the safe-integer ceiling 9007199254740991');
+  return Number(v);
+}
+
 // Validate a local string table's shape BEFORE StringTable.parse casts a
 // Uint32Array over it (T-1..T-4): the offsets array must be in bounds, the
 // count/blob must fit, offsets must be non-decreasing, and the trailing
@@ -118,11 +140,11 @@ export class Reader {
     if (reserved1 !== 0)
       throw new ReaderError('R_RESERVED_NONZERO', 'header reserved1 at offset 36 must be 0, got ' + reserved1);
 
-    this._schemaBlockOff  = Number(this._dv.getBigUint64(8, true));
-    this._metadataOff     = Number(this._dv.getBigUint64(16, true));   // 0 = no metadata block; M7+ zone maps
-    this._shardDirOff     = Number(this._dv.getBigUint64(24, true));
+    this._schemaBlockOff  = u64(this._dv, 8, 'schema_block_off');
+    this._metadataOff     = u64(this._dv, 16, 'metadata_off');   // 0 = no metadata block; M7+ zone maps
+    this._shardDirOff     = u64(this._dv, 24, 'shard_directory_off');
     this._shardCount      = this._dv.getUint32(32, true);
-    this._totalRows       = Number(this._dv.getBigUint64(40, true));
+    this._totalRows       = u64(this._dv, 40, 'total_rows');
     this._formatVersion   = version;
 
     const len = this._buffer.byteLength;
@@ -180,7 +202,10 @@ export class Reader {
       const laneKind    = this._bytes[descOff + 4];
       const flags       = this._bytes[descOff + 5];
       const reserved2   = this._dv.getUint16(descOff + 6, true);
-      const nameStrOff  = Number(this._dv.getBigUint64(descOff + 8, true));
+      const nameStrOff  = u64(this._dv, descOff + 8, 'field ' + i + ' name_str_off');
+      // reserved3 is compared as BigInt vs 0n below and never Number()-cast, so
+      // it needs no u64() narrowing guard -- a value past 2^53 is caught by the
+      // reserved-must-be-zero check, not by precision loss (BS-05).
       const reserved3   = this._dv.getBigUint64(descOff + 16, true);
 
       if (flags !== 0) throw new ReaderError('R_BAD_FIELD_FLAGS', 'field ' + i + ' has non-zero flags (v2+ reserved)');
@@ -212,14 +237,14 @@ export class Reader {
     let rowSum = 0;
     for (let i = 0; i < this._shardCount; i++) {
       const entryOff = off + i * SHARD_ENTRY_BYTES;
-      const payloadOff    = Number(this._dv.getBigUint64(entryOff + 0, true));
+      const payloadOff    = u64(this._dv, entryOff + 0, 'shard ' + i + ' payload_off');
       const payloadLen    = this._dv.getUint32(entryOff + 8, true);
       const rowCount      = this._dv.getUint32(entryOff + 12, true);
       const minReaderVer  = this._dv.getUint16(entryOff + 16, true);
       const shardFlags    = this._dv.getUint16(entryOff + 18, true);
       const shardReserved = this._dv.getUint32(entryOff + 20, true);
-      const localStrOff   = Number(this._dv.getBigUint64(entryOff + 24, true));
-      const localStrLen   = Number(this._dv.getBigUint64(entryOff + 32, true));
+      const localStrOff   = u64(this._dv, entryOff + 24, 'shard ' + i + ' local_string_off');
+      const localStrLen   = u64(this._dv, entryOff + 32, 'shard ' + i + ' local_string_len');
       if (minReaderVer > READER_VERSION) {
         throw new ReaderError('R_SHARD_VERSION_TOO_NEW',
           'shard ' + i + ' requires reader version ' + minReaderVer);
@@ -396,13 +421,24 @@ export class Reader {
     return i;
   }
 
+  // Cold: the BS-32 row-range refusal body, kept out of get()'s hot frame.
+  _badRow(rowIdx) {
+    throw new ReaderError('R_ROW_OUT_OF_RANGE',
+      'rowIdx ' + rowIdx + ' out of range [0, ' + this._totalRows + ')');
+  }
+
   // Random-access get across all shards. Returns the field's decoded value:
   //   F64 lanes -> number
   //   U32 lanes -> JS string (resolved via the shard's local string table)
   // Not zero-alloc; intended for testing/introspection.
   get(rowIdx, fieldName) {
-    let remaining = rowIdx;
     const fieldIdx = this.fieldIndex(fieldName);
+    // BS-32 row-range door. `(rowIdx >>> 0) !== rowIdx` rejects negative,
+    // fractional, NaN and >= 2^32 in one compare; a whole ArrayBuffer cannot hold
+    // 2^32 rows, so the totalRows compare fires first for every reachable
+    // container. Throw body hoisted to the cold _badRow helper.
+    if ((rowIdx >>> 0) !== rowIdx || rowIdx >= this._totalRows) this._badRow(rowIdx);
+    let remaining = rowIdx;
     const field = this._schema.fields[fieldIdx];
     for (let s = 0; s < this._shards.length; s++) {
       const shard = this._shards[s];

package/src/Split.js

@@ -37,7 +37,7 @@ import { Reader, ReaderError } from './Reader.js';
 import { StringTable } from './StringTable.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 const LF = 0x0A;
 const CONTAINER_HEADER_BYTES = 48;
@@ -219,6 +219,11 @@ export function mergeContainers(containers) {
   // Compute the target layout. Schema block is copied from container 0 verbatim.
   // Reader gives us schemaBlockOff and shardDirOff; the schema-block byte range
   // is [schemaBlockOff, shardDirOff). We reuse this slice.
+  // These Number(getBigUint64) reads carry NO u64 narrowing guard by design
+  // (BS-05 dominance): `new Reader(...)` at :193 above parsed every part first,
+  // and Reader's own guarded u64() over the same header bytes (schema_block_off,
+  // shard_directory_off) already threw R_OFFSET_TOO_LARGE for any over-2^53
+  // value. Split mints nothing here -- an untriggerable code would be dead.
   const src0 = parts[0];
   const dv0 = new DataView(src0.buffer, src0.byteOffset, src0.byteLength);
   const schemaBlockOff = Number(dv0.getBigUint64(8, true));
@@ -383,7 +388,10 @@ function _schemasEqual(a, b) {
   return true;
 }
 
-// Look up the source string-table byte length for a shard.
+// Look up the source string-table byte length for a shard. No u64 narrowing
+// guard here (BS-05 dominance): `reader` is a fully-parsed Reader, so Reader's
+// guarded u64() over this same local_string_len field already threw
+// R_OFFSET_TOO_LARGE for any over-2^53 value during construction.
 function _sourceStringTableLen(reader, shardIdx) {
   const dv = new DataView(reader.buffer);
   const entryOff = reader.shardDirectoryOffset + shardIdx * SHARD_ENTRY_BYTES;

package/src/StringTable.js

@@ -19,8 +19,26 @@
 // Entry 0 is always the empty string: the table reserves it in the constructor
 // and at every reset(), so an absent U32 row cell (which is 0) decodes as ""
 // rather than aliasing the shard's first-interned string (SPEC 3.3, SPEC 7).
+//
+// Serialization ceilings (BS-31). serialize() writes entry_count and
+// blob_length as u32; a table that grew past either would silently wrap on
+// setUint32 and mint a corrupt-but-in-bounds container. Two cold guards in
+// _insertNew fail closed instead:
+//   ST_BLOB_OVERFLOW  - the blob would exceed the u32 blob_length ceiling, OR
+//                       the entry count would exceed the entry_count ceiling
+//                       (one code, two messages -- both name their ceiling).
+// The doubling clamps in _growBlob / _growOffsets cap allocation at the
+// ceiling so a crossing write ALWAYS re-enters the grow branch where the guard
+// lives; guard placement is then provably complete (no path reaches serialize
+// with an out-of-range count/blob). Zero hot cost: the guards read only inside
+// the cold grow arms.
+//
+// __setStringTableLimits({blobBytes, entryCount}) lowers the ceilings for a
+// cheap gated crossing on the real grow path and returns the previous pair.
+// TEST-ONLY: not re-exported from index.js, absent from the .d.ts and docs,
+// and carrying no semver guarantee.
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 const EMPTY_SLOT = 0xFFFFFFFF;   // MUST be unsigned; typed-array reads are unsigned
 const INITIAL_BLOB_BYTES  = 64 * 1024;
@@ -29,6 +47,26 @@ const INITIAL_HASH_CAP    = 2048;      // load factor target 50%
 const HASH_MAX_LOAD_NUM   = 1;         // 50% load factor: num/den = 1/2
 const HASH_MAX_LOAD_DEN   = 2;
 
+// u32 serialization ceilings. blob_length is a u32 (max 4294967295). entry_count
+// is a u32, but the offsets array carries entry_count+1 slots (the sentinel), so
+// the last addressable entry_count is 4294967294 to keep offsets_len a valid u32.
+// `let`, not `const`: __setStringTableLimits lowers them for gated crossings.
+let LIMIT_BLOB = 4294967295;
+let LIMIT_ENTRIES = 4294967294;
+
+// TEST-ONLY seam (BS-31). Assigns whichever of {blobBytes, entryCount} are
+// present, returns the previous pair. Not part of the public API.
+export function __setStringTableLimits(next) {
+  const prev = { blobBytes: LIMIT_BLOB, entryCount: LIMIT_ENTRIES };
+  if (next && next.blobBytes !== undefined) LIMIT_BLOB = next.blobBytes;
+  if (next && next.entryCount !== undefined) LIMIT_ENTRIES = next.entryCount;
+  return prev;
+}
+
+export class StringTableError extends Error {
+  constructor(code, msg) { super(msg); this.code = code; this.name = 'StringTableError'; }
+}
+
 // Reserved entry 0. A zero-length range: no blob bytes, no growth, one hash
 // slot. Module-level so reset() reserves without allocating.
 const EMPTY = new Uint8Array(0);
@@ -94,10 +132,23 @@ export class StringTable {
 
   _insertNew(bytes, from, to, slot) {
     const n = to - from;
-    // Grow blob if needed
-    if (this._blobLen + n > this._blob.length) this._growBlob(n);
-    // Grow offsets if needed (leave room for +1 sentinel)
-    if (this._count + 1 >= this._offsets.length) this._growOffsets();
+    // Grow blob if needed. The clamp keeps _blob.length <= LIMIT_BLOB, so any
+    // write pushing past the ceiling necessarily re-enters this branch and is
+    // caught by the cold guard (fail closed -- serialize would wrap otherwise).
+    if (this._blobLen + n > this._blob.length) {
+      if (this._blobLen + n > LIMIT_BLOB)
+        throw new StringTableError('ST_BLOB_OVERFLOW',
+          'string blob ' + this._blobLen + ' + ' + n + ' would exceed the u32 blob_length ceiling ' + LIMIT_BLOB);
+      this._growBlob(n);
+    }
+    // Grow offsets if needed (leave room for +1 sentinel). Same completeness
+    // argument via the _growOffsets clamp on the entry_count ceiling.
+    if (this._count + 1 >= this._offsets.length) {
+      if (this._count >= LIMIT_ENTRIES)
+        throw new StringTableError('ST_BLOB_OVERFLOW',
+          'string table entry count ' + this._count + ' would exceed the entry_count ceiling ' + LIMIT_ENTRIES);
+      this._growOffsets();
+    }
 
     const idx = this._count;
     this._offsets[idx] = this._blobLen;
@@ -123,13 +174,18 @@ export class StringTable {
   _growBlob(needed) {
     let cap = this._blob.length;
     while (cap < this._blobLen + needed) cap *= 2;
+    // Clamp so allocated length never exceeds the u32 ceiling (see _insertNew).
+    if (cap > LIMIT_BLOB) cap = LIMIT_BLOB;
     const nb = new Uint8Array(cap);
     nb.set(this._blob);
     this._blob = nb;
   }
 
   _growOffsets() {
-    const nb = new Uint32Array(this._offsets.length * 2);
+    let len = this._offsets.length * 2;
+    // Clamp so the offsets array (entry_count + 1 slots) never exceeds a valid u32.
+    if (len > LIMIT_ENTRIES + 1) len = LIMIT_ENTRIES + 1;
+    const nb = new Uint32Array(len);
     nb.set(this._offsets);
     this._offsets = nb;
   }

package/src/Tokenizer.js

@@ -30,7 +30,7 @@
 
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 const U32_MAX = 4294967295;
 const TOKENIZER_OPTS = {

package/src/Writer.js

@@ -23,6 +23,21 @@
 // String fields intern via the per-shard StringTable which allocates only on
 // unique inserts (bounded by cardinality, not row count).
 //
+// Shard-byte budget (D2): a shard rolls when its row count reaches the maxRows
+// ceiling OR when payload_bytes + string_table_bytes reaches targetShardBytes.
+// The byte term is CHUNK-INDEPENDENT: it reads only _currentShardRowCount,
+// rowStride, _targetShardBytes and _stringTableBytes -- the first three are
+// chunk-invariant and _stringTableBytes is a pure function of the unique
+// interned byte ranges in arrival order (= logical record order, which the
+// Tokenizer delivers identically for any input chunking). No chunk-local state
+// (absOffset, buffer boundaries) enters, so re-chunked input yields byte-
+// identical containers (the t0 law). A single row or string that alone exceeds
+// targetShardBytes still gets its own oversized shard (maxRows has a
+// Math.max(1, ...) floor and the budget runs post-write); payload_len stays
+// u32-honest via the targetShardBytes constructor guard. A schema with no U32
+// lane keeps a zero string budget and rolls on rows only -- byte-identical to
+// the pre-D2 baseline.
+//
 // Error codes (stable):
 //   W_TOP_LEVEL_NOT_OBJECT     - top-level value is not an object
 //   W_NESTED_UNSUPPORTED       - nested object/array inside a record
@@ -43,7 +58,7 @@
 import { StringTable } from './StringTable.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 const U32_MAX = 4294967295;
 // Post-finalize sentinel for _recordDepth. Chosen = 2 so every post-finalize
@@ -105,6 +120,12 @@ export class WriterError extends Error {
   }
 }
 
+// Shared decoder for sample-window key names (BS-27 floor). onKey fires once per
+// key per sample record; a fresh TextDecoder each time is pure garbage. One
+// module-scope instance, reused -- decode() is stateless across calls. The other
+// TextDecoders in the codebase are per-open/lazy and stay as they are.
+const KEY_DECODER = new TextDecoder();
+
 function hashBytes(bytes, from, to) {
   let h = 0x811c9dc5 | 0;
   for (let i = from; i < to; i++) {
@@ -226,6 +247,15 @@ export class Writer {
     this._shards = [];                  // { bytes, rowCount, stringTableBytes }
     this._totalRows = 0;
 
+    // D2 shard-byte budget (zero-alloc, chunk-independent). _stringTableBytes is
+    // the serialized byte length the CURRENT shard's string table would emit RIGHT
+    // NOW; _maxInternedIdx is the highest interned index seen this shard, so a new
+    // unique arrival (strIdx > _maxInternedIdx) is detected in one compare. Both
+    // recomputed only in the cold unique-string arm (bounded by cardinality, not
+    // rows) and reset at each shard finalize. Real values are set at freeze.
+    this._stringTableBytes = 0;
+    this._maxInternedIdx = 0;
+
     // Per-record parse state
     this._recordDepth = 0;
     this._currentKeyBytes = null;
@@ -285,7 +315,7 @@ export class Writer {
       // buffer for the value bytes before onNumber/onString fires, so we cannot
       // defer decoding. String allocation here is expected — bounded to the
       // sample window; steady-state post-freeze remains zero-alloc.
-      this._currentKeyName = new TextDecoder().decode(bytes.subarray(from, to));
+      this._currentKeyName = KEY_DECODER.decode(bytes.subarray(from, to));
     }
   }
 
@@ -343,8 +373,17 @@ export class Writer {
       const lane = this._fieldLaneKinds[idx];
       if (lane !== LANE_U32) throw new WriterError('W_LANE_MISMATCH',
         'field ' + this._schema.fields[idx].name + ' is not a U32 lane but got a string');
-      const strIdx = this._currentShardStringTable().intern(bytes, from, to);
+      const st = this._currentShardStringTable();
+      const strIdx = st.intern(bytes, from, to);
       this._rowValueSlotsU32[idx] = strIdx;
+      // D2 (cold arm, unique arrivals only): recompute the serialized string-table
+      // byte length. serialize() emits (8 + 4*(count+1) + blobLen), 8-padded, and
+      // count = strIdx + 1 for a fresh entry, so the padded form is
+      // (16 + 4*strIdx + blobLen) rounded up to 8 -- exact vs StringTable.serialize.
+      if (strIdx > this._maxInternedIdx) {
+        this._maxInternedIdx = strIdx;
+        this._stringTableBytes = (16 + 4 * strIdx + st.blobLen + 7) & ~7;
+      }
     } else {
       if (!this._currentKeyName) return;
       this._sample.setString(this._currentKeyName, bytes, from, to);
@@ -452,6 +491,19 @@ export class Writer {
       this._fieldLaneKinds[i] = fields[i].laneKind;
       this._fieldOffsets[i] = fields[i].offsetInRow;
     }
+    // R7 floor: O(1) field-name resolution. hash -> field index, or an array of
+    // indices when two names collide (Array.isArray in the cold arm). Built once
+    // at freeze; _lookupFieldIdx replaces its O(F) scan with one Map.get plus the
+    // mandatory byte-equal confirm, so a hash collision still resolves correctly.
+    const fieldHashMap = new Map();
+    for (let i = 0; i < fields.length; i++) {
+      const h = this._fieldNameHashes[i];
+      const existing = fieldHashMap.get(h);
+      if (existing === undefined) fieldHashMap.set(h, i);
+      else if (Array.isArray(existing)) existing.push(i);
+      else fieldHashMap.set(h, [existing, i]);
+    }
+    this._fieldHashMap = fieldHashMap;
     this._rowValueSlotsF64 = new Float64Array(fields.length);
     this._rowValueSlotsU32 = new Uint32Array(fields.length);
     this._currentShardMaxRows = Math.max(1, Math.floor(this._targetShardBytes / rowStride));
@@ -467,14 +519,27 @@ export class Writer {
     this._trackedFieldToPos = new Int32Array(fields.length);
     for (let i = 0; i < fields.length; i++) this._trackedFieldToPos[i] = -1;
     for (let t = 0; t < tracked.length; t++) this._trackedFieldToPos[tracked[t]] = t;
+
+    // D2 budget baseline. A fresh string table always carries reserved entry 0,
+    // so a U32-lane shard with no real strings still serializes to 16 bytes; a
+    // schema with no U32 lane emits NO table (0 bytes, EMPTY_STRING_TABLE_BYTES).
+    // Seeding _stringTableBytes to 0 for the F64-only case keeps such containers
+    // byte-identical to the pre-D2 baseline (the roll then fires on rows only via
+    // the maxRows term, which is <= the byte term for a zero string budget).
+    this._maxInternedIdx = 0;
+    this._stringTableBytes = this._hasU32 ? 16 : 0;
   }
 
   _lookupFieldIdx(bytes, from, to) {
     const h = hashBytes(bytes, from, to);
-    const hashes = this._fieldNameHashes;
+    const hit = this._fieldHashMap.get(h);
+    if (hit === undefined) return -1;
     const names = this._fieldNamesUtf8;
-    for (let i = 0; i < hashes.length; i++) {
-      if (hashes[i] === h && bytesEqual(bytes, from, to, names[i])) return i;
+    // Hit path (the common case): a single index. Byte-confirm and return.
+    if (typeof hit === 'number') return bytesEqual(bytes, from, to, names[hit]) ? hit : -1;
+    // Cold: >= 2 field names share this hash; byte-confirm each candidate.
+    for (let k = 0; k < hit.length; k++) {
+      if (bytesEqual(bytes, from, to, names[hit[k]])) return hit[k];
     }
     return -1;
   }
@@ -511,7 +576,15 @@ export class Writer {
       }
       this._currentShardRowCount++;
       this._totalRows++;
-      if (this._currentShardRowCount >= this._currentShardMaxRows) this._finalizeCurrentShard();
+      // D2 roll: fire on the row ceiling OR when payload + string-table bytes
+      // reach the target. The row-ceiling term fires first for any F64-only shard
+      // (string budget 0, maxRows = floor(target/stride) <= target/stride), so
+      // such containers stay byte-identical to baseline; string shards can roll
+      // earlier. Zero allocation -- one add + compare per row.
+      const n = this._currentShardRowCount;
+      if (n >= this._currentShardMaxRows ||
+          n * this._schema.rowStride + this._stringTableBytes >= this._targetShardBytes)
+        this._finalizeCurrentShard();
     } else {
       const endOff = this._source !== null ? this._source.absOffset : this._absOffset;
       const consumed = Math.max(1, endOff - this._recordStartByteOffset);
@@ -564,6 +637,10 @@ export class Writer {
       bytes: copy,
       rowCount: this._currentShardRowCount,
       stringTableBytes: stBytes,
+      // D2 audit: the incrementally-tracked budget at this finalize. MUST equal
+      // stBytes.length (the emitted directory local_string_len). Cross-checked by
+      // Ceilings.test.js via __stringTableBudgetAudit; not part of the public API.
+      budgetTracked: this._stringTableBytes,
       mins: shardMins,
       maxes: shardMaxes,
     });
@@ -574,6 +651,9 @@ export class Writer {
     // Reset string table for next shard (per-shard independence)
     st.reset();
     this._perShardStringTable = st;
+    // Reset the D2 budget trackers alongside st.reset() (per-shard independence).
+    this._maxInternedIdx = 0;
+    this._stringTableBytes = this._hasU32 ? 16 : 0;
   }
 
   _drainSampleToShards() {
@@ -607,6 +687,11 @@ export class Writer {
       const trackedIdx = this._trackedFieldToPos;
       const mins = this._currentShardMins;
       const maxes = this._currentShardMaxes;
+      // D2: honour the same string-aware byte budget the post-freeze hot path
+      // uses, so a sample-drained container and a post-freeze one agree on shard
+      // boundaries for identical logical input. writtenRows may finish below
+      // chunkRows when the string table pushes the shard over target.
+      let writtenRows = chunkRows;
       for (let r = 0; r < chunkRows; r++) {
         const rowOff = r * stride;
         for (let f = 0; f < fields.length; f++) {
@@ -625,14 +710,24 @@ export class Writer {
             const strBytes = sampleTable.bytesAt(sampleIdx);
             const newIdx = strBytes ? dstTable.intern(strBytes, 0, strBytes.length) : 0;
             dv.setUint32(fo, newIdx, true);
+            if (newIdx > this._maxInternedIdx) {
+              this._maxInternedIdx = newIdx;
+              this._stringTableBytes = (16 + 4 * newIdx + dstTable.blobLen + 7) & ~7;
+            }
           }
         }
+        const done = r + 1;
+        if (done < chunkRows &&
+            done * stride + this._stringTableBytes >= this._targetShardBytes) {
+          writtenRows = done;
+          break;
+        }
       }
-      this._currentShardRowCount = chunkRows;
-      this._totalRows += chunkRows;
+      this._currentShardRowCount = writtenRows;
+      this._totalRows += writtenRows;
       this._finalizeCurrentShard();     // serializes dstTable, resets in-place for the next iter
-      srcRow += chunkRows;
-      rowsRemaining -= chunkRows;
+      srcRow += writtenRows;
+      rowsRemaining -= writtenRows;
     }
 
     this._sample = null;
@@ -836,4 +931,17 @@ export class Writer {
   get schema()      { return this._schema; }
   get totalRows()   { return this._totalRows; }
   get shardCount()  { return this._shards.length + (this._currentShardRowCount > 0 ? 1 : 0); }
+
+  // @internal TEST-ONLY (D2): per-shard [{ tracked, emitted }] string-table byte
+  // lengths. tracked is the incrementally-maintained budget at each finalize;
+  // emitted is the serialized local_string_len that reached the container. They
+  // MUST be equal at every shard -- Ceilings.test.js proves it, closing the
+  // formula-drift bug class. No semver guarantee; absent from the public API.
+  __stringTableBudgetAudit() {
+    const out = new Array(this._shards.length);
+    for (let i = 0; i < this._shards.length; i++) {
+      out[i] = { tracked: this._shards[i].budgetTracked, emitted: this._shards[i].stringTableBytes.length };
+    }
+    return out;
+  }
 }

package/src/index.js

@@ -12,7 +12,7 @@
 import { Tokenizer, TokenizerError } from './Tokenizer.js';
 import { Writer, WriterError } from './Writer.js';
 import { Reader, ReaderError } from './Reader.js';
-import { StringTable } from './StringTable.js';
+import { StringTable, StringTableError } from './StringTable.js';
 import { PreserveTokenizer, PreserveTokenizerError } from './PreserveTokenizer.js';
 import { PreserveWriter, PreserveWriterError } from './PreserveWriter.js';
 import { PreserveReader, PreserveReaderError } from './PreserveReader.js';
@@ -23,12 +23,12 @@ export {
   Tokenizer, TokenizerError,
   Writer, WriterError,
   Reader, ReaderError,
-  StringTable,
+  StringTable, StringTableError,
   PreserveTokenizer, PreserveTokenizerError,
   PreserveWriter, PreserveWriterError,
   PreserveReader, PreserveReaderError,
 };
-export const VERSION = '1.4.1';
+export const VERSION = '1.5.0';
 
 const encoder = new TextEncoder();
 

package/types/StringTable.d.ts

@@ -3,6 +3,11 @@
 
 export const VERSION: string;
 
+export class StringTableError extends Error {
+  readonly code: string;
+  readonly name: 'StringTableError';
+}
+
 export class StringTable {
   constructor();
   /** Intern a byte range. Zero-alloc on the hit path. Returns u32 index. */

package/types/index.d.ts

@@ -13,7 +13,7 @@ export { Reader, ReaderError } from './Reader.d.ts';
 export type {
   ShardHandle, StringTableView, Bounds, FindShardsOptions,
 } from './Reader.d.ts';
-export { StringTable } from './StringTable.d.ts';
+export { StringTable, StringTableError } from './StringTable.d.ts';
 export { PreserveTokenizer, PreserveTokenizerError } from './PreserveTokenizer.d.ts';
 export type { PreserveSink, PreserveTokenizerOptions } from './PreserveTokenizer.d.ts';
 export { PreserveWriter, PreserveWriterError } from './PreserveWriter.d.ts';