@zakkster/lite-bake-stream 1.2.0 → 1.3.1

package/CHANGELOG.md

@@ -2,6 +2,46 @@
 
 All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.3.1] -- 2026-09-01
+
+M3 remainder (BS-05) -- the UTF-8 door ends the promise-A/behavior-B split with a measured decision. Option A (validate at the tokenizer, refuse invalid UTF-8 with `E_INVALID_UTF8`) was built, proven correctness-green (460/460, torture fast 42/42), and rejected on its pre-committed budget: unicode-heavy throughput regressed 229.7 -> 213.1 MB/s (-7.2%, over the 3% ceiling), because the baseline scan bulk-copies `>= 0x80` bytes with zero inspection so validation is pure added cost, while ASCII stayed within noise. SPEC 5.5 is instead rewritten to match the long-standing behavior (documented-permissive), and the dead code is removed. No bytes move at runtime. See decisions/0006-utf8-door.md for the before/after numbers and the rejected option. Suite: 453 tests / 453 pass / 0 fail / 0 todo. Torture: 42/42 fast, 44/44 full, arrayBuffers growth 0. Inverted mutation check: all 49 battery and pin tests pass on baseline src as well -- zero behavior change, proven.
+
+### Changed
+
+- SPEC 5.5 and SPEC 7's U32 row rewritten to documented-permissive: string bytes between the quotes are stored byte-exact through U32 lanes regardless of UTF-8 validity; invalid sequences are not a parse error. `get()`/`getString()` decode with replacement (U+FFFD); `shardStringTable(s).bytesAt(i)` returns the exact stored bytes, the schema-mode twin of `PreserveReader.getBytes(i)`. This is a documentation-and-dead-code change only -- no runtime behavior changed, no new public surface (the accessor already existed).
+
+### Fixed
+
+- BS-05: the SPEC no longer promises a UTF-8 parse error the tokenizer never enforced, and `Tokenizer.js` no longer declares an unreachable `E_INVALID_UTF8` code. The byte-exact string contract is now stated truthfully: invalid bytes round-trip byte-exact and decode with U+FFFD, in schema mode exactly as in preserve mode. Lone-surrogate escapes still refuse with `E_UNPAIRED_SURROGATE` (an escape-grammar rule, unrelated to input-UTF-8 validity).
+
+## [1.3.0] -- 2026-09-01
+
+M2 -- a container is verified at the door or refused at the door. The four readers (`Reader`, `PreserveReader`, `RangeReader`, `MultiReader`) now agree with each other and with the `Writer` about the bytes between them; every malformed container names its refusal with a code, and a standalone SPEC byte-checker (reads SPEC.md offsets only, imports nothing from `src/`) is the contract's executable form. All new validation runs at open/parse/construction time -- the `get`/`syncRange`/`findShards` row loops and both tokenizer byte loops are byte-identical. Suite: 432 tests / 431 pass / 0 fail / 1 todo. Torture: 37/37 fast, 39/39 full, arrayBuffers growth 0. Mutation check: all 52 door tests fail on source revert; all 23 behavior-unchanged pins hold.
+
+### Added
+
+- New stable refusal codes: `R_RESERVED_NONZERO` (a reserved header/descriptor/shard field is non-zero), `R_BAD_FOOTER` (footer `magic_end` or `footer_len` malformed), `R_BAD_METADATA` (a non-zero `metadata_off` pointing at an unparseable zone-map segment), and `M_TOO_MANY_ROWS` (a `MultiReader` union whose cumulative row count exceeds 2^53).
+- `src/Views.js` (internal, not exported): a shared `toContainerBuffer` helper so `deserialize`, `Reader.fromBuffer`, and `PreserveReader.fromBuffer` resolve caller bytes by one rule -- a full-buffer view is unwrapped in place, a partial view is copied.
+
+### Changed
+
+- Malformed containers that previously opened, half-opened, or failed later with a raw `RangeError` now refuse at the door with a code. Reserved bytes, footer shape, structural bounds (`row_count * row_stride <= payload_len`, `offset_in_row + lane_bytes <= row_stride`, name-blob and payload extents, monotonic string-table offsets with a `blob_length` sentinel, `total_rows == sum(row_count)`) are all checked at parse time. Every byte-prefix of a valid container either opens fully or throws an `R_`/`M_` code -- no raw `RangeError`, no clamped subarray, no silent short read.
+- `findShards`/`shardBounds` with a field name absent from the schema now throw `R_UNKNOWN_FIELD` (`M_UNKNOWN_FIELD` on `MultiReader`) instead of silently returning all shards / `null`. A field present in the schema but untracked by zone maps still returns all shards / `null` (decisions/0005).
+- F64-only shards no longer emit a 16-byte empty local string table -- such containers are `16 * shardCount` bytes smaller and their ShardEntries carry `local_string_off = 0` / `local_string_len = 0` (SPEC 3.4). `mergeContainers` applies the same rule to its output even when fed a pre-M2 container that violated it.
+- `RangeReader.open` performs one additional 16-byte footer fetch (open-phase fetch count 5 -> 6 for a container with zone maps), so a wrong-footer or short container is refused before the schema fetch.
+- `MultiReader` cumulative row/shard offsets moved `Uint32Array` -> `Float64Array`, so a union totaling more than 2^32 rows locates rows correctly instead of wrapping to `null`.
+
+### Fixed
+
+- BS-06: a truncated preserve container no longer returns a clamped short `subarray`; the shard directory is validated at open.
+- BS-08: reserved header/descriptor/shard bytes and the footer are now enforced; a lying `metadata_off` throws `R_BAD_METADATA` instead of silently degrading or throwing a raw `RangeError`.
+- BS-09: `Reader.fromBuffer` now respects a `Uint8Array`'s `byteOffset`/`byteLength`, copying a partial view instead of reading a sibling's bytes from a pooled buffer.
+- BS-10: `RangeReader.open` on a preserve-mode container throws `R_WRONG_MODE` naming `PreserveReader`, instead of misreading it as schema mode.
+- BS-15: F64-only shards emit no local string table, matching SPEC 3.4's 0/0 rule.
+- BS-16: `MultiReader` over sub-readers totaling more than 2^32 rows addresses every row exactly; beyond 2^53 it refuses with `M_TOO_MANY_ROWS`.
+- BS-17: one unknown-field policy across `get`/`findShards`/`shardBounds` on all readers (decisions/0005).
+- Sink-poison rider (M1 deferred): a sink callback that throws mid-`feed()`/`end()` now leaves the tokenizer poisoned; the next `feed()`/`end()` throws `E_POISONED`. Both tokenizers, via an armed-flag pattern with zero `try`/`catch` and two boolean stores per chunk.
+
 ## [1.2.0] -- 2026-08-31
 
 M1 -- the write path refuses what it cannot store. Eight findings closed by a set of doors that all run at construction, freeze, or terminal transition; the byte and record hot paths are unchanged (proven by the torture t6 gate, numbers within noise of M0). Suite: 359 tests / 358 pass / 0 fail / 1 todo. Torture: 37/37 fast, 39/39 full, arrayBuffers growth 0. Mutation check: all 63 door tests fail on source revert; all 13 behavior-unchanged pins hold.

package/README.md

@@ -13,7 +13,7 @@
 
 > Streaming byte-level JSON compiler for [`@zakkster/lite-bake`](https://github.com/PeshoVurtoleta/lite-bake). Zero-GC, tree-shakeable, gigabyte-scale.
 
-**Status:** v1.2.0. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak with zero GC and byte-exact preservation across 590 million cells.
+**Status:** v1.3.1. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak with zero GC and byte-exact preservation across 590 million cells.
 
 ## Two modes, one API
 

package/SPEC.md

@@ -274,7 +274,7 @@ The boundary is exercised explicitly by `test/NumericBoundary.test.js`, which pi
 
 ### 5.5 String semantics (v1)
 
-- Input MUST be valid UTF-8. Invalid sequences are a parse error.
+- String bytes between the quotes are stored byte-exact through U32 lanes, regardless of UTF-8 validity. Invalid sequences are **not** a parse error; the prior rule declaring them one (and its `E_INVALID_UTF8` code) is withdrawn. `get()` / `getString()` decode with replacement (U+FFFD); `shardStringTable(s).bytesAt(i)` returns the exact stored bytes, as `PreserveReader.getBytes(i)` does in preserve mode.
 - All standard JSON escapes are supported: `\"`, `\\`, `\/`, `\b`, `\f`, `\n`, `\r`, `\t`, `\uXXXX`.
 - Surrogate pairs (`\uD800`..`\uDBFF` followed by `\uDC00`..`\uDFFF`) are combined and emitted as UTF-8.
 - Unpaired surrogates are a parse error.
@@ -334,7 +334,7 @@ Given source NDJSON `S` and a schema `Σ` (either explicit or inferred), the LBK
 | :-- | :-- | :-- |
 | JSON number in fast-path domain (≤15 sig digits, `|exp| ≤ 22`) | F64 | **bit-exact IEEE 754 double** |
 | JSON number in slow-path domain | F64 | within 1 ULP of the correctly-rounded value |
-| JSON string (any valid UTF-8) | U32 | **byte-exact UTF-8 sequence** |
+| JSON string (any bytes, valid UTF-8 or not) | U32 | **byte-exact bytes; validity not asserted** |
 | JSON `true` / `false` / `null` | F64 | `1` / `0` / `0` (documented coercion) |
 | absent field (missing in source) | F64 | `0` (default) |
 | absent field | U32 | `""` (index 0 is the reserved empty-string entry) |

package/llms.txt

@@ -8,13 +8,13 @@ Ingest gigabyte-scale JSON (top-level array or NDJSON) into the `lite-bake` LBK1
 
 ## Status
 
-v1.2.0 — stable. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak (M1 MacBook Pro): 98.37M rows, 4.89 GB container, zero major GC, zero minor GC, 499 KB total heap allocation, 590.21M cells verified byte-exact, zero mismatches. Tokenizer benches at 222-237 MB/s (~55% of JSON.parse, with no object graph allocated).
+v1.3.1 — stable. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak (M1 MacBook Pro): 98.37M rows, 4.89 GB container, zero major GC, zero minor GC, 499 KB total heap allocation, 590.21M cells verified byte-exact, zero mismatches. Tokenizer benches at 222-237 MB/s (~55% of JSON.parse, with no object graph allocated).
 
 Public API follows semver from 1.0.0. Future additions (I64 lane, columnar payload mode, container-level string table) land via the format's forward-compat seams -- `min_reader_version` on ShardEntry, reserved FieldDescriptor flags, the `metadata_off` block wrapper -- without a format_version bump.
 
 See SPEC.md for the LBK1 container format, section 3.6 for zone maps, section 4.3 for the reserved field flags.
 
-## Public API (v1.2.0)
+## Public API (v1.3.1)
 
 Two ingest modes share one top-level API:
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@zakkster/lite-bake-stream",
-  "version": "1.2.0",
+  "version": "1.3.1",
   "description": "Streaming byte-level JSON to lite-bake binary compiler. Zero-GC, tree-shakeable, gigabyte-scale.",
   "type": "module",
   "sideEffects": false,
@@ -109,6 +109,10 @@
   "engines": {
     "node": ">=18"
   },
+  "funding": {
+    "type": "github",
+    "url": "https://github.com/sponsors/PeshoVurtoleta"
+  },
   "publishConfig": {
     "access": "public"
   },

package/src/FileIngest.js

@@ -27,7 +27,7 @@ import { PreserveWriter } from './PreserveWriter.js';
 import { PreserveReader } from './PreserveReader.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 const U32_MAX = 4294967295;
 const INGEST_OPTS = {

package/src/MultiReader.js

@@ -21,10 +21,11 @@
 // Error codes:
 //   M_EMPTY               - no readers passed
 //   M_SCHEMA_MISMATCH     - schemas differ across readers
-//   M_UNKNOWN_FIELD       - .get() / .fieldIndex() called with unknown name
+//   M_UNKNOWN_FIELD       - .get() / .fieldIndex() / findShards / shardBounds with unknown name
 //   M_ROW_OUT_OF_RANGE    - rowIdx >= totalRows
+//   M_TOO_MANY_ROWS       - cumulative row count exceeds Number.MAX_SAFE_INTEGER
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 export class MultiReaderError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'MultiReaderError'; }
@@ -60,15 +61,26 @@ export class MultiReader {
 
     // Cumulative row offsets and shard offsets. Row rowIdx belongs to reader
     // r iff _rowOffsets[r] <= rowIdx < _rowOffsets[r+1].
+    // Float64Array (not Uint32Array): cumulative row/shard offsets can exceed
+    // 2^32 across many large sub-readers. F64 is exact to 2^53 and stays flat
+    // and monotonic, so the linear scan in _locateRow/_locateShard is unchanged
+    // apart from the element type (BS-16).
     let cumRows = 0, cumShards = 0;
-    this._rowOffsets   = new Uint32Array(readers.length + 1);
-    this._shardOffsets = new Uint32Array(readers.length + 1);
+    this._rowOffsets   = new Float64Array(readers.length + 1);
+    this._shardOffsets = new Float64Array(readers.length + 1);
     for (let i = 0; i < readers.length; i++) {
       this._rowOffsets[i]   = cumRows;
       this._shardOffsets[i] = cumShards;
       cumRows   += readers[i].totalRows;
       cumShards += readers[i].shardCount;
     }
+    // Boundary is 2^53 (BRIEF T6 / A8): total_rows == 2^53 keeps the largest
+    // row index at 2^53-1 (= MAX_SAFE_INTEGER), still exactly representable;
+    // beyond 2^53 an index collides with its neighbour in F64.
+    if (cumRows > 2 ** 53) {
+      throw new MultiReaderError('M_TOO_MANY_ROWS',
+        'cumulative row count ' + cumRows + ' exceeds 2^53; row indices past 2^53 are not exactly addressable in F64');
+    }
     this._rowOffsets[readers.length]   = cumRows;
     this._shardOffsets[readers.length] = cumShards;
     this._totalRows = cumRows;
@@ -129,6 +141,8 @@ export class MultiReader {
   // reader; results are concatenated and re-numbered into the global shard
   // index space.
   findShards(fieldName, opts) {
+    if (!this._fieldIndex.has(fieldName))
+      throw new MultiReaderError('M_UNKNOWN_FIELD', 'no field named ' + JSON.stringify(fieldName));
     const out = [];
     for (let r = 0; r < this._readers.length; r++) {
       const localShards = this._readers[r].findShards(fieldName, opts);
@@ -139,6 +153,8 @@ export class MultiReader {
   }
 
   shardBounds(globalShardIdx, fieldName) {
+    if (!this._fieldIndex.has(fieldName))
+      throw new MultiReaderError('M_UNKNOWN_FIELD', 'no field named ' + JSON.stringify(fieldName));
     const loc = this._locateShard(globalShardIdx);
     if (!loc) return null;
     return this._readers[loc.readerIdx].shardBounds(loc.localShard, fieldName);

package/src/PreserveReader.js

@@ -16,11 +16,27 @@
 // The Uint8Array returned by getBytes is a live subarray view of the container
 // buffer. It is valid for as long as the Reader instance holds the container.
 // Do NOT mutate it, and do NOT retain it past the Reader's lifetime.
+//
+// Error codes:
+//   R_BAD_MAGIC              - header magic mismatch
+//   R_UNSUPPORTED_VERSION    - format_version > 1
+//   R_UNSUPPORTED_ENDIAN     - endian byte is not LE
+//   R_WRONG_MODE             - schema-mode container fed to the PreserveReader
+//   R_BAD_FLAGS              - unknown header flag bits set
+//   R_TRUNCATED              - a structure claims bytes past the buffer end
+//   R_RESERVED_NONZERO       - a reserved header/shard field is non-zero
+//   R_BAD_FOOTER             - footer magic_end or footer_len is malformed
+//   R_INVALID                - a structure is internally inconsistent but in-bounds
+//   R_SHARD_VERSION_TOO_NEW  - a shard's min_reader_version > this reader
+//   R_ROW_OUT_OF_RANGE       - rowIdx >= totalRows
+
+import { toContainerBuffer } from './Views.js';
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 const CONTAINER_HEADER_BYTES = 48;
 const SHARD_ENTRY_BYTES = 40;
+const FOOTER_BYTES = 16;
 
 export class PreserveReaderError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'PreserveReaderError'; }
@@ -28,16 +44,7 @@ export class PreserveReaderError extends Error {
 
 export class PreserveReader {
   static fromBuffer(input) {
-    if (input instanceof ArrayBuffer) return new PreserveReader(input);
-    if (input instanceof Uint8Array) {
-      if (input.byteOffset === 0 && input.byteLength === input.buffer.byteLength) {
-        return new PreserveReader(input.buffer);
-      }
-      const copy = new Uint8Array(input.byteLength);
-      copy.set(input);
-      return new PreserveReader(copy.buffer);
-    }
-    throw new TypeError('PreserveReader.fromBuffer: expected ArrayBuffer or Uint8Array');
+    return new PreserveReader(toContainerBuffer(input, 'PreserveReader.fromBuffer'));
   }
 
   constructor(buffer) {
@@ -49,6 +56,7 @@ export class PreserveReader {
     this._bytes = new Uint8Array(buffer);
     this._decoder = new TextDecoder('utf-8', { fatal: false });
     this._parseHeader();
+    this._parseFooter();
     this._parseShardDirectory();
   }
 
@@ -72,17 +80,40 @@ export class PreserveReader {
     if (flags & ~0x01) {
       throw new PreserveReaderError('R_BAD_FLAGS', 'unknown flag bits set: 0x' + flags.toString(16));
     }
+    const reserved1 = this._dv.getUint32(36, true);
+    if (reserved1 !== 0)
+      throw new PreserveReaderError('R_RESERVED_NONZERO', 'header reserved1 at offset 36 must be 0, got ' + reserved1);
 
     this._schemaBlockOff = Number(this._dv.getBigUint64(8, true));
     this._metadataOff    = Number(this._dv.getBigUint64(16, true));
     this._shardDirOff    = Number(this._dv.getBigUint64(24, true));
     this._shardCount     = this._dv.getUint32(32, true);
-    // 4 bytes reserved at 36
     this._totalRows      = Number(this._dv.getBigUint64(40, true));
 
     if (this._schemaBlockOff !== 0) {
       throw new PreserveReaderError('R_INVALID', 'preserve container has non-zero schema_block_off');
     }
+    if (this._metadataOff !== 0) {
+      throw new PreserveReaderError('R_INVALID', 'preserve container has non-zero metadata_off (SPEC 3.8)');
+    }
+    const len = this._buffer.byteLength;
+    if (this._shardDirOff < CONTAINER_HEADER_BYTES || this._shardDirOff >= len)
+      throw new PreserveReaderError('R_TRUNCATED', 'shard_directory_off ' + this._shardDirOff + ' out of range [48, ' + len + ')');
+  }
+
+  // Footer (SPEC 3.7): magic_end '1KBL' + footer_len >= 16.
+  _parseFooter() {
+    const len = this._buffer.byteLength;
+    const footerOff = len - FOOTER_BYTES;
+    const b = this._bytes;
+    if (b[footerOff + 8] !== 0x31 || b[footerOff + 9] !== 0x4B ||
+        b[footerOff + 10] !== 0x42 || b[footerOff + 11] !== 0x4C)
+      throw new PreserveReaderError('R_BAD_FOOTER', 'footer magic_end at offset ' + (footerOff + 8) + ' is not 1KBL');
+    const footerLen = this._dv.getUint32(footerOff + 12, true);
+    if (footerLen < FOOTER_BYTES)
+      throw new PreserveReaderError('R_BAD_FOOTER', 'footer_len ' + footerLen + ' is less than the minimum ' + FOOTER_BYTES);
+    if (footerLen > len - CONTAINER_HEADER_BYTES)
+      throw new PreserveReaderError('R_BAD_FOOTER', 'footer_len ' + footerLen + ' exceeds the container body size');
   }
 
   _parseShardDirectory() {
@@ -100,21 +131,46 @@ export class PreserveReader {
       const payloadLen = this._dv.getUint32(entryOff + 8, true);
       const rowCount   = this._dv.getUint32(entryOff + 12, true);
       const minReaderVer = this._dv.getUint16(entryOff + 16, true);
+      const shardFlags   = this._dv.getUint16(entryOff + 18, true);
+      const shardReserved = this._dv.getUint32(entryOff + 20, true);
+      const localStrOff  = Number(this._dv.getBigUint64(entryOff + 24, true));
+      const localStrLen  = Number(this._dv.getBigUint64(entryOff + 32, true));
       if (minReaderVer > 1) {
         throw new PreserveReaderError('R_SHARD_VERSION_TOO_NEW',
           'shard ' + i + ' requires reader version ' + minReaderVer);
       }
+      if (shardFlags !== 0) throw new PreserveReaderError('R_RESERVED_NONZERO', 'shard ' + i + ' flags must be 0, got ' + shardFlags);
+      if (shardReserved !== 0) throw new PreserveReaderError('R_RESERVED_NONZERO', 'shard ' + i + ' reserved must be 0, got ' + shardReserved);
+      if (localStrOff !== 0 || localStrLen !== 0)
+        throw new PreserveReaderError('R_INVALID', 'shard ' + i + ' local_string_* must be 0 in preserve mode (SPEC 3.8)');
+      if (payloadOff < CONTAINER_HEADER_BYTES || payloadOff + payloadLen > this._buffer.byteLength)
+        throw new PreserveReaderError('R_TRUNCATED', 'shard ' + i + ' payload [' + payloadOff + ', ' + (payloadOff + payloadLen) + ') out of range');
       // Trailing u32 offset table.
       const offsetTableOff = payloadOff + payloadLen - rowCount * 4;
       if (offsetTableOff < payloadOff) {
         throw new PreserveReaderError('R_INVALID',
           'shard ' + i + ' payload_len (' + payloadLen + ') too small for offset table of ' + rowCount + ' rows');
       }
+      const blobLen = offsetTableOff - payloadOff;
+      // Offset-table walk: offsets[0] must be 0, non-decreasing, last <= blobLen.
+      if (rowCount > 0) {
+        let prev = this._dv.getUint32(offsetTableOff, true);
+        if (prev !== 0)
+          throw new PreserveReaderError('R_INVALID', 'shard ' + i + ' offset table[0] is ' + prev + ', must be 0');
+        for (let r = 1; r < rowCount; r++) {
+          const cur = this._dv.getUint32(offsetTableOff + r * 4, true);
+          if (cur < prev)
+            throw new PreserveReaderError('R_INVALID', 'shard ' + i + ' offset table not monotonic at record ' + r);
+          prev = cur;
+        }
+        if (prev > blobLen)
+          throw new PreserveReaderError('R_INVALID', 'shard ' + i + ' offset table last entry ' + prev + ' exceeds blob length ' + blobLen);
+      }
       shards[i] = {
         payloadOff,
         payloadLen,
         rowCount,
-        blobLen: offsetTableOff - payloadOff,
+        blobLen,
         offsetTableOff,
         firstRow: cumulativeRow,
         endRow: cumulativeRow + rowCount,

package/src/PreserveTokenizer.js

@@ -32,7 +32,7 @@
 
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 const U32_MAX = 4294967295;
 const PRESERVE_TOKENIZER_OPTS = {
@@ -88,23 +88,28 @@ export class PreserveTokenizer {
   }
 
   feed(chunk) {
-    if (this._poisoned) throw new PreserveTokenizerError('E_POISONED', 'tokenizer poisoned by a previous error; construct a new instance');
+    if (this._poisoned) throw new PreserveTokenizerError('E_POISONED', 'tokenizer poisoned by a previous error or an in-flight feed(); construct a new instance');
     if (this._ended) throw new PreserveTokenizerError('E_ENDED', 'tokenizer already ended; construct a new instance');
     if (!(chunk instanceof Uint8Array)) {
       throw new TypeError('PreserveTokenizer.feed: expected Uint8Array');
     }
-    if (chunk.length === 0) return;
+    // D3 armed-flag: a sink throw inside _scan unwinds with the flag set,
+    // poisoning the instance. Cleared at every clean exit (empty chunk and end).
+    this._poisoned = true;
+    if (chunk.length === 0) { this._poisoned = false; return; }
     this._ensureCapacity(this._bufLen + chunk.length);
     this._buf.set(chunk, this._bufLen);
     const scanStart = this._bufLen;
     this._bufLen += chunk.length;
     this._scan(scanStart);
     this._compact();
+    this._poisoned = false;
   }
 
   end() {
-    if (this._poisoned) throw new PreserveTokenizerError('E_POISONED', 'tokenizer poisoned by a previous error; construct a new instance');
+    if (this._poisoned) throw new PreserveTokenizerError('E_POISONED', 'tokenizer poisoned by a previous error or an in-flight feed(); construct a new instance');
     if (this._ended) throw new PreserveTokenizerError('E_ENDED', 'tokenizer already ended; construct a new instance');
+    this._poisoned = true;   // D3: armed until the clean exit below
     if (this._bufLen > this._recordStart) {
       // There's pending content that never terminated.
       if (this._depth !== 0 || this._inString || this._escape) {
@@ -119,6 +124,7 @@ export class PreserveTokenizer {
       this._recordStart = this._bufLen;
     }
     this._ended = true;
+    this._poisoned = false;   // clean end; a throwing onRecord above stays poisoned
   }
 
   _scan(from) {

package/src/PreserveWriter.js

@@ -18,7 +18,7 @@
 
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 export class PreserveWriterError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'PreserveWriterError'; }

package/src/RangeReader.js

@@ -9,9 +9,9 @@
 // table offsets exist precisely so a client can fetch only what it needs.
 //
 // RangeReader implements that story:
-//   1. On open, fetch the header (32 bytes) + schema block + shard directory.
-//      That gives us the whole schema and every shard's byte extents without
-//      touching a single row.
+//   1. On open, fetch the header (48 bytes) + footer + schema block + shard
+//      directory. That gives us the whole schema and every shard's byte extents
+//      without touching a single row.
 //   2. On get(rowIdx, fieldName), locate the shard containing rowIdx, fetch
 //      its payload + local string table in ONE range request (they are
 //      contiguous by SPEC 3.4), cache it, decode.
@@ -32,11 +32,17 @@
 //   R_UNSUPPORTED_LANE / R_SHARD_VERSION_TOO_NEW / R_TRUNCATED / R_UNKNOWN_FIELD
 //   R_ADAPTER_SHORT_READ   -- adapter returned fewer bytes than requested
 //   R_ROW_OUT_OF_RANGE     -- rowIdx >= totalRows
+//   R_WRONG_MODE           -- preserve-mode container fed to the schema RangeReader
+//   R_BAD_FLAGS            -- unknown header flag bits set
+//   R_RESERVED_NONZERO     -- a reserved header/descriptor/shard field is non-zero
+//   R_BAD_FOOTER           -- footer magic_end or footer_len is malformed
+//   R_BAD_METADATA         -- metadata_off non-zero but the zone-map segment is unparseable
+//   R_INVALID              -- a structure is internally inconsistent but in-bounds
 
 import { StringTable } from './StringTable.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 const CONTAINER_HEADER_BYTES = 48;
 const SHARD_ENTRY_BYTES = 40;
@@ -50,11 +56,40 @@ const U32_MAX = 4294967295;
 const RANGE_READER_OPTS = { maxCachedShards: { t: 'int', min: 0, max: U32_MAX } };
 const HTTP_ADAPTER_OPTS = { fetch: { t: 'fn' } };
 
+const CONTAINER_FOOTER_BYTES = 16;
+
+function laneBytesOf(k) { return k === LANE_F64 ? 8 : (k === LANE_U32 ? 4 : 0); }
+
 export class RangeReaderError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'RangeReaderError'; }
 }
 function raiseRange(code, msg) { throw new RangeReaderError(code, msg); }
 
+// Validate a local string table's shape before StringTable.parse casts a
+// Uint32Array over it (T-1..T-4). `bytes` is the already-fetched shard slice;
+// `off` and `len` locate the table within it.
+function validateStringTable(bytes, off, len, label) {
+  if (len < 8)
+    throw new RangeReaderError('R_TRUNCATED', label + ' string table shorter than its 8-byte header');
+  const dv = new DataView(bytes.buffer, bytes.byteOffset + off, len);
+  const entryCount = dv.getUint32(0, true);
+  const blobLen = dv.getUint32(4, true);
+  if (8 + (entryCount + 1) * 4 + blobLen > len)
+    throw new RangeReaderError('R_TRUNCATED', label + ' string table claims ' + entryCount +
+      ' entries + ' + blobLen + ' blob bytes past its ' + len + '-byte extent');
+  let prev = dv.getUint32(8, true);
+  if (prev !== 0)
+    throw new RangeReaderError('R_INVALID', label + ' string table offsets[0] is ' + prev + ', must be 0');
+  for (let i = 1; i <= entryCount; i++) {
+    const cur = dv.getUint32(8 + i * 4, true);
+    if (cur < prev)
+      throw new RangeReaderError('R_INVALID', label + ' string table offsets not monotonic at index ' + i);
+    prev = cur;
+  }
+  if (prev !== blobLen)
+    throw new RangeReaderError('R_INVALID', label + ' string table sentinel ' + prev + ' != blob_length ' + blobLen);
+}
+
 // ----- Adapters --------------------------------------------------------------
 
 // Uses fetch() with Range headers. Content-Length is discovered on open() via
@@ -169,7 +204,7 @@ export class RangeReader {
       throw new RangeReaderError('R_TRUNCATED', 'adapter size smaller than header');
     }
 
-    // Step 1: header (32 bytes) tells us schema + shard-dir offsets.
+    // Step 1: header (48 bytes) tells us schema + shard-dir offsets.
     const headerBytes = await this.adapter.fetch(0, CONTAINER_HEADER_BYTES);
     const hdrDv = new DataView(headerBytes.buffer, headerBytes.byteOffset, headerBytes.byteLength);
     if (headerBytes[0] !== 0x4C || headerBytes[1] !== 0x42 || headerBytes[2] !== 0x4B || headerBytes[3] !== 0x31) {
@@ -179,21 +214,48 @@ export class RangeReader {
     if (version > 1) throw new RangeReaderError('R_UNSUPPORTED_VERSION', 'format_version=' + version);
     const endian = headerBytes[6];
     if (endian !== 1) throw new RangeReaderError('R_UNSUPPORTED_ENDIAN', 'BE payloads not implemented in v1 reader');
+    const flags = headerBytes[7];
+    if (flags & 0x01) {
+      throw new RangeReaderError('R_WRONG_MODE',
+        'container is preserve-mode; RangeReader is unsupported in preserve mode (SPEC 3.8) -- use PreserveReader');
+    }
+    if (flags & ~0x01) {
+      throw new RangeReaderError('R_BAD_FLAGS', 'unknown flag bits set in header byte 7: 0x' + flags.toString(16));
+    }
+    const reserved1 = hdrDv.getUint32(36, true);
+    if (reserved1 !== 0)
+      throw new RangeReaderError('R_RESERVED_NONZERO', 'header reserved1 at offset 36 must be 0, got ' + reserved1);
 
     this._schemaBlockOff = Number(hdrDv.getBigUint64(8, true));
     this._metadataOff    = Number(hdrDv.getBigUint64(16, true));
     this._shardDirOff    = Number(hdrDv.getBigUint64(24, true));
     this._shardCount     = hdrDv.getUint32(32, true);
-    // 4 bytes reserved at 36
     this._totalRows      = Number(hdrDv.getBigUint64(40, true));
 
+    const size = this.adapter.size;
+    if (this._schemaBlockOff < CONTAINER_HEADER_BYTES || this._schemaBlockOff >= size)
+      throw new RangeReaderError('R_TRUNCATED', 'schema_block_off ' + this._schemaBlockOff + ' out of range [48, ' + size + ')');
+    if (this._shardDirOff < CONTAINER_HEADER_BYTES || this._shardDirOff >= size)
+      throw new RangeReaderError('R_TRUNCATED', 'shard_directory_off ' + this._shardDirOff + ' out of range [48, ' + size + ')');
+    if (this._metadataOff !== 0 && (this._metadataOff < CONTAINER_HEADER_BYTES || this._metadataOff >= size))
+      throw new RangeReaderError('R_TRUNCATED', 'metadata_off ' + this._metadataOff + ' out of range {0} u [48, ' + size + ')');
+    if (this._shardCount * SHARD_ENTRY_BYTES > size - CONTAINER_HEADER_BYTES)
+      throw new RangeReaderError('R_TRUNCATED', 'shard_count ' + this._shardCount + ' exceeds the bytes available for a shard directory');
+
+    // Footer: one 16-byte suffix fetch. Refuse a wrong-footer container before
+    // spending fetches on the schema. Gated on a body big enough to hold a
+    // header and a footer that do not overlap.
+    if (size >= CONTAINER_HEADER_BYTES + CONTAINER_FOOTER_BYTES) await this._loadFooter();
+
     // Step 2: schema block. We know its start but not its length; compute
     // upper bound: schemaBlockOff .. shardDirOff.
+    if (this._shardDirOff <= this._schemaBlockOff)
+      throw new RangeReaderError('R_TRUNCATED', 'shard_directory_off ' + this._shardDirOff + ' not after schema_block_off ' + this._schemaBlockOff);
     const schemaBlockLen = this._shardDirOff - this._schemaBlockOff;
     const schemaBytes = await this.adapter.fetch(this._schemaBlockOff, schemaBlockLen);
     this._parseSchema(schemaBytes);
 
-    // Step 3: shard directory (fixed size = shardCount * 32).
+    // Step 3: shard directory (fixed size = shardCount * 40).
     const dirBytes = await this.adapter.fetch(this._shardDirOff, this._shardCount * SHARD_ENTRY_BYTES);
     this._parseShardDirectory(dirBytes);
 
@@ -210,21 +272,43 @@ export class RangeReader {
     }
   }
 
+  async _loadFooter() {
+    const size = this.adapter.size;
+    const footer = await this.adapter.fetch(size - CONTAINER_FOOTER_BYTES, CONTAINER_FOOTER_BYTES);
+    if (footer[8] !== 0x31 || footer[9] !== 0x4B || footer[10] !== 0x42 || footer[11] !== 0x4C)
+      throw new RangeReaderError('R_BAD_FOOTER', 'footer magic_end is not 1KBL');
+    const fDv = new DataView(footer.buffer, footer.byteOffset, footer.byteLength);
+    const footerLen = fDv.getUint32(12, true);
+    if (footerLen < CONTAINER_FOOTER_BYTES)
+      throw new RangeReaderError('R_BAD_FOOTER', 'footer_len ' + footerLen + ' is less than the minimum ' + CONTAINER_FOOTER_BYTES);
+    if (footerLen > size - CONTAINER_HEADER_BYTES)
+      throw new RangeReaderError('R_BAD_FOOTER', 'footer_len ' + footerLen + ' exceeds the container body size');
+  }
+
   async _loadZoneMaps() {
-    // Fetch the segment header first (16 bytes) so we can size the rest.
+    // Fetch the segment header first (16 bytes) so we can size the rest. A
+    // non-zero metadata_off asserts a parseable segment; any failure is
+    // corruption (D1), not absence.
+    if (this._metadataOff + 16 > this.adapter.size)
+      throw new RangeReaderError('R_BAD_METADATA', 'zone maps segment header runs past the container');
     const hdr = await this.adapter.fetch(this._metadataOff, 16);
-    if (hdr[0] !== 0x30 || hdr[1] !== 0x5A || hdr[2] !== 0x4D || hdr[3] !== 0x31) return;
+    if (hdr[0] !== 0x30 || hdr[1] !== 0x5A || hdr[2] !== 0x4D || hdr[3] !== 0x31)
+      throw new RangeReaderError('R_BAD_METADATA', 'zone maps magic is not ZM01');
     const hdrDv = new DataView(hdr.buffer, hdr.byteOffset, hdr.byteLength);
     const shardCount = hdrDv.getUint32(4, true);
     const T          = hdrDv.getUint32(8, true);
-    if (shardCount !== this._shardCount) {
-      throw new RangeReaderError('R_TRUNCATED', 'zone maps shard_count mismatch');
-    }
+    const reserved0  = hdrDv.getUint32(12, true);
+    if (shardCount !== this._shardCount)
+      throw new RangeReaderError('R_BAD_METADATA', 'zone maps shard_count ' + shardCount + ' != header ' + this._shardCount);
+    if (reserved0 !== 0)
+      throw new RangeReaderError('R_BAD_METADATA', 'zone maps reserved0 must be 0, got ' + reserved0);
     if (T === 0) return;
     // Fetch the rest in one range: field table + pad + mins + maxes.
     const fieldTableLen = T * 2;
     const fieldTablePad = (8 - (fieldTableLen & 7)) & 7;
     const restLen = fieldTableLen + fieldTablePad + shardCount * T * 8 * 2;
+    if (this._metadataOff + 16 + restLen > this.adapter.size)
+      throw new RangeReaderError('R_BAD_METADATA', 'zone maps segment runs past the container');
     const rest = await this.adapter.fetch(this._metadataOff + 16, restLen);
     const restDv = new DataView(rest.buffer, rest.byteOffset, rest.byteLength);
     const tracked = new Array(T);
@@ -232,7 +316,7 @@ export class RangeReader {
     for (let t = 0; t < T; t++) {
       const schemaFieldIdx = restDv.getUint16(t * 2, true);
       if (schemaFieldIdx >= this._schema.fields.length) {
-        throw new RangeReaderError('R_TRUNCATED', 'zone maps references field index ' + schemaFieldIdx);
+        throw new RangeReaderError('R_BAD_METADATA', 'zone maps references field index ' + schemaFieldIdx + ' past the schema');
       }
       tracked[t] = schemaFieldIdx;
       fieldToPos.set(this._schema.fields[schemaFieldIdx].name, t);
@@ -250,13 +334,20 @@ export class RangeReader {
 
   _parseSchema(bytes) {
     const dv = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
+    const total = bytes.byteLength;
+    if (total < 8) throw new RangeReaderError('R_TRUNCATED', 'schema block shorter than its 8-byte header');
     const fieldCount = dv.getUint32(0, true);
     const rowStride  = dv.getUint32(4, true);
+    if (rowStride < 1) throw new RangeReaderError('R_INVALID', 'schema row_stride is 0');
     const descOff = 8;
     const descBytes = fieldCount * FIELD_DESCRIPTOR_BYTES;
+    if (descOff + descBytes + 4 > total)
+      throw new RangeReaderError('R_TRUNCATED', 'schema descriptors run past the fetched schema block');
     const nameBlobLenOff = descOff + descBytes;
     const nameBlobLen = dv.getUint32(nameBlobLenOff, true);
     const nameBlobOff = nameBlobLenOff + 4;
+    if (nameBlobOff + nameBlobLen > total)
+      throw new RangeReaderError('R_TRUNCATED', 'schema name blob runs past the fetched schema block');
     const decoder = new TextDecoder('utf-8');
     const fields = new Array(fieldCount);
     for (let i = 0; i < fieldCount; i++) {
@@ -265,10 +356,19 @@ export class RangeReader {
       const offsetInRow = dv.getUint16(off + 2, true);
       const laneKind    = bytes[off + 4];
       const flags       = bytes[off + 5];
+      const reserved2   = dv.getUint16(off + 6, true);
       const nameStrOff  = Number(dv.getBigUint64(off + 8, true));
+      const reserved3   = dv.getBigUint64(off + 16, true);
       if (flags !== 0) throw new RangeReaderError('R_BAD_FIELD_FLAGS', 'field ' + i + ' has non-zero flags');
+      if (reserved2 !== 0) throw new RangeReaderError('R_RESERVED_NONZERO', 'field ' + i + ' reserved2 must be 0, got ' + reserved2);
+      if (reserved3 !== 0n) throw new RangeReaderError('R_RESERVED_NONZERO', 'field ' + i + ' reserved3 must be 0, got ' + reserved3);
       if (laneKind !== LANE_F64 && laneKind !== LANE_U32)
         throw new RangeReaderError('R_UNSUPPORTED_LANE', 'field ' + i + ' lane_kind=' + laneKind);
+      if (nameLen > 255) throw new RangeReaderError('R_INVALID', 'field ' + i + ' name_len ' + nameLen + ' exceeds 255');
+      if (offsetInRow + laneBytesOf(laneKind) > rowStride)
+        throw new RangeReaderError('R_INVALID', 'field ' + i + ' offset_in_row ' + offsetInRow + ' + lane bytes exceeds row_stride ' + rowStride);
+      if (nameStrOff + nameLen > nameBlobLen)
+        throw new RangeReaderError('R_TRUNCATED', 'field ' + i + ' name range past the name blob');
       const nb = bytes.subarray(nameBlobOff + nameStrOff, nameBlobOff + nameStrOff + nameLen);
       fields[i] = { name: decoder.decode(nb), laneKind, offsetInRow };
     }
@@ -278,6 +378,8 @@ export class RangeReader {
   _parseShardDirectory(bytes) {
     const dv = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
     const shards = new Array(this._shardCount);
+    const size = this.adapter.size;
+    const rowStride = this._schema.rowStride;
     let cumulativeRow = 0;
     for (let i = 0; i < this._shardCount; i++) {
       const off = i * SHARD_ENTRY_BYTES;
@@ -285,12 +387,25 @@ export class RangeReader {
       const payloadLen   = dv.getUint32(off + 8, true);
       const rowCount     = dv.getUint32(off + 12, true);
       const minReaderVer = dv.getUint16(off + 16, true);
-      // 2 bytes flags at 18, 4 bytes reserved at 20
+      const shardFlags   = dv.getUint16(off + 18, true);
+      const shardReserved = dv.getUint32(off + 20, true);
       const localStrOff  = Number(dv.getBigUint64(off + 24, true));
       const localStrLen  = Number(dv.getBigUint64(off + 32, true));
       if (minReaderVer > READER_VERSION)
         throw new RangeReaderError('R_SHARD_VERSION_TOO_NEW',
           'shard ' + i + ' requires reader version ' + minReaderVer);
+      if (shardFlags !== 0) throw new RangeReaderError('R_RESERVED_NONZERO', 'shard ' + i + ' flags must be 0, got ' + shardFlags);
+      if (shardReserved !== 0) throw new RangeReaderError('R_RESERVED_NONZERO', 'shard ' + i + ' reserved must be 0, got ' + shardReserved);
+      if (payloadOff < CONTAINER_HEADER_BYTES)
+        throw new RangeReaderError('R_INVALID', 'shard ' + i + ' payload_off ' + payloadOff + ' overlaps the header');
+      if (payloadOff + payloadLen > size)
+        throw new RangeReaderError('R_TRUNCATED', 'shard ' + i + ' payload past the container size ' + size);
+      if (rowCount * rowStride > payloadLen)
+        throw new RangeReaderError('R_INVALID', 'shard ' + i + ' row_count ' + rowCount + ' * row_stride ' + rowStride + ' exceeds payload_len ' + payloadLen);
+      if (localStrLen === 0 && localStrOff !== 0)
+        throw new RangeReaderError('R_INVALID', 'shard ' + i + ' has local_string_len 0 but local_string_off ' + localStrOff);
+      if (localStrLen > 0 && localStrOff + localStrLen > size)
+        throw new RangeReaderError('R_TRUNCATED', 'shard ' + i + ' string table past the container size ' + size);
       shards[i] = {
         payloadOff, payloadLen, rowCount, localStrOff, localStrLen,
         firstRow: cumulativeRow,
@@ -298,6 +413,8 @@ export class RangeReader {
       };
       cumulativeRow += rowCount;
     }
+    if (cumulativeRow !== this._totalRows)
+      throw new RangeReaderError('R_INVALID', 'shard row_count sum ' + cumulativeRow + ' != header total_rows ' + this._totalRows);
     this._shards = shards;
   }
 
@@ -354,6 +471,7 @@ export class RangeReader {
     let stringTable = null;
     if (s.localStrLen > 0) {
       const stBytes = combined.subarray(s.payloadLen, s.payloadLen + s.localStrLen);
+      validateStringTable(stBytes, 0, s.localStrLen, 'shard ' + shardIdx);
       stringTable = StringTable.parse(stBytes, 0);
     }
     const record = {
@@ -448,6 +566,8 @@ export class RangeReader {
   // the container has no zone maps or the field is not tracked. Synchronous
   // -- zone maps were fetched during open().
   shardBounds(shardIdx, fieldName) {
+    if (!this._fieldIndex.has(fieldName))
+      throw new RangeReaderError('R_UNKNOWN_FIELD', 'no field named ' + JSON.stringify(fieldName));
     if (!this._zoneMapsTrackedFields) return null;
     const t = this._zoneMapsFieldToPos.get(fieldName);
     if (t === undefined) return null;
@@ -460,6 +580,8 @@ export class RangeReader {
   // When no zone maps are present or the field is not tracked, returns every
   // shard (query planner must fall back to full scan).
   findShards(fieldName, opts) {
+    if (!this._fieldIndex.has(fieldName))
+      throw new RangeReaderError('R_UNKNOWN_FIELD', 'no field named ' + JSON.stringify(fieldName));
     const all = () => { const a = new Array(this._shardCount); for (let i = 0; i < this._shardCount; i++) a[i] = i; return a; };
     if (!this._zoneMapsTrackedFields) return all();
     const t = this._zoneMapsFieldToPos.get(fieldName);

package/src/Reader.js

@@ -10,15 +10,22 @@
 //   R_BAD_MAGIC              - header magic mismatch
 //   R_UNSUPPORTED_VERSION    - format_version > 1
 //   R_UNSUPPORTED_ENDIAN     - endian byte is neither LE nor BE
+//   R_WRONG_MODE             - preserve-mode container fed to the schema Reader
+//   R_BAD_FLAGS              - unknown header flag bits set
 //   R_BAD_FIELD_FLAGS        - a FieldDescriptor.flags is non-zero (v2+ reserved)
 //   R_UNSUPPORTED_LANE       - lane_kind is not F64 or U32
 //   R_SHARD_VERSION_TOO_NEW  - a shard's min_reader_version > this reader
-//   R_TRUNCATED              - container ends before an expected structure
-//   R_UNKNOWN_FIELD          - get() called with an unknown field name
+//   R_TRUNCATED              - a structure claims bytes past the buffer end
+//   R_RESERVED_NONZERO       - a reserved header/descriptor/shard field is non-zero
+//   R_BAD_FOOTER             - footer magic_end or footer_len is malformed
+//   R_BAD_METADATA           - metadata_off is non-zero but the zone-map segment is unparseable
+//   R_INVALID                - a structure is internally inconsistent but in-bounds
+//   R_UNKNOWN_FIELD          - get()/findShards()/shardBounds() called with an unknown field name
 
 import { StringTable } from './StringTable.js';
+import { toContainerBuffer } from './Views.js';
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 const CONTAINER_HEADER_BYTES = 48;
 const SHARD_ENTRY_BYTES = 40;
@@ -29,16 +36,43 @@ const LANE_F64 = 1;
 const LANE_U32 = 3;
 const READER_VERSION = 1;
 
+function laneBytesOf(k) { return k === LANE_F64 ? 8 : (k === LANE_U32 ? 4 : 0); }
+
 export class ReaderError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'ReaderError'; }
 }
 
+// Validate a local string table's shape BEFORE StringTable.parse casts a
+// Uint32Array over it (T-1..T-4): the offsets array must be in bounds, the
+// count/blob must fit, offsets must be non-decreasing, and the trailing
+// sentinel must equal blob_length. A lie here is otherwise a raw RangeError
+// from `new Uint32Array(buffer, off, ...)` or a silent out-of-range read.
+function validateStringTable(bytes, off, len, label) {
+  if (len < 8)
+    throw new ReaderError('R_TRUNCATED', label + ' string table shorter than its 8-byte header');
+  const dv = new DataView(bytes.buffer, bytes.byteOffset + off, len);
+  const entryCount = dv.getUint32(0, true);
+  const blobLen = dv.getUint32(4, true);
+  const offsetsBytes = (entryCount + 1) * 4;
+  if (8 + offsetsBytes + blobLen > len)
+    throw new ReaderError('R_TRUNCATED', label + ' string table claims ' + entryCount +
+      ' entries + ' + blobLen + ' blob bytes past its ' + len + '-byte extent');
+  let prev = dv.getUint32(8, true);
+  if (prev !== 0)
+    throw new ReaderError('R_INVALID', label + ' string table offsets[0] is ' + prev + ', must be 0');
+  for (let i = 1; i <= entryCount; i++) {
+    const cur = dv.getUint32(8 + i * 4, true);
+    if (cur < prev)
+      throw new ReaderError('R_INVALID', label + ' string table offsets not monotonic at index ' + i);
+    prev = cur;
+  }
+  if (prev !== blobLen)
+    throw new ReaderError('R_INVALID', label + ' string table sentinel ' + prev + ' != blob_length ' + blobLen);
+}
+
 export class Reader {
-  static fromBuffer(bufferOrArrayBuffer) {
-    const buffer = bufferOrArrayBuffer instanceof ArrayBuffer
-      ? bufferOrArrayBuffer
-      : bufferOrArrayBuffer.buffer;
-    return new Reader(buffer);
+  static fromBuffer(input) {
+    return new Reader(toContainerBuffer(input, 'Reader.fromBuffer'));
   }
 
   constructor(buffer) {
@@ -46,9 +80,10 @@ export class Reader {
     this._dv = new DataView(buffer);
     this._bytes = new Uint8Array(buffer);
     this._parseHeader();
+    this._parseFooter();
     this._parseSchema();
     this._parseShardDirectory();
-    this._parseZoneMaps();      // M7 — no-op if metadata_off is 0
+    this._parseZoneMaps();      // M7 -- no-op if metadata_off is 0
     this._buildFieldIndex();
   }
 
@@ -79,13 +114,42 @@ export class Reader {
     }
     this._flags = flags;
 
+    const reserved1 = this._dv.getUint32(36, true);
+    if (reserved1 !== 0)
+      throw new ReaderError('R_RESERVED_NONZERO', 'header reserved1 at offset 36 must be 0, got ' + reserved1);
+
     this._schemaBlockOff  = Number(this._dv.getBigUint64(8, true));
     this._metadataOff     = Number(this._dv.getBigUint64(16, true));   // 0 = no metadata block; M7+ zone maps
     this._shardDirOff     = Number(this._dv.getBigUint64(24, true));
     this._shardCount      = this._dv.getUint32(32, true);
-    // 4 bytes reserved at 36
     this._totalRows       = Number(this._dv.getBigUint64(40, true));
     this._formatVersion   = version;
+
+    const len = this._buffer.byteLength;
+    if (this._schemaBlockOff < CONTAINER_HEADER_BYTES || this._schemaBlockOff >= len)
+      throw new ReaderError('R_TRUNCATED', 'schema_block_off ' + this._schemaBlockOff + ' out of range [48, ' + len + ')');
+    if (this._shardDirOff < CONTAINER_HEADER_BYTES || this._shardDirOff >= len)
+      throw new ReaderError('R_TRUNCATED', 'shard_directory_off ' + this._shardDirOff + ' out of range [48, ' + len + ')');
+    if (this._metadataOff !== 0 && (this._metadataOff < CONTAINER_HEADER_BYTES || this._metadataOff >= len))
+      throw new ReaderError('R_TRUNCATED', 'metadata_off ' + this._metadataOff + ' out of range {0} u [48, ' + len + ')');
+    if (this._shardCount * SHARD_ENTRY_BYTES > len - CONTAINER_HEADER_BYTES)
+      throw new ReaderError('R_TRUNCATED', 'shard_count ' + this._shardCount + ' exceeds the bytes available for a shard directory');
+  }
+
+  // Footer (SPEC 3.7): magic_end 'LBK1'-reversed = '1KBL' at footer_off+8,
+  // footer_len >= 16 at footer_off+12. Runs once, at construction.
+  _parseFooter() {
+    const len = this._buffer.byteLength;
+    const footerOff = len - FOOTER_BYTES;
+    const b = this._bytes;
+    if (b[footerOff + 8] !== 0x31 || b[footerOff + 9] !== 0x4B ||
+        b[footerOff + 10] !== 0x42 || b[footerOff + 11] !== 0x4C)
+      throw new ReaderError('R_BAD_FOOTER', 'footer magic_end at offset ' + (footerOff + 8) + ' is not 1KBL');
+    const footerLen = this._dv.getUint32(footerOff + 12, true);
+    if (footerLen < FOOTER_BYTES)
+      throw new ReaderError('R_BAD_FOOTER', 'footer_len ' + footerLen + ' is less than the minimum ' + FOOTER_BYTES);
+    if (footerLen > len - CONTAINER_HEADER_BYTES)
+      throw new ReaderError('R_BAD_FOOTER', 'footer_len ' + footerLen + ' exceeds the container body size');
   }
 
   _parseSchema() {
@@ -93,6 +157,7 @@ export class Reader {
     if (off + 8 > this._buffer.byteLength) throw new ReaderError('R_TRUNCATED', 'schema header truncated');
     const fieldCount  = this._dv.getUint32(off + 0, true);
     const rowStride   = this._dv.getUint32(off + 4, true);
+    if (rowStride < 1) throw new ReaderError('R_INVALID', 'schema row_stride is 0');
     const descriptorsOff = off + 8;
     const descriptorsBytes = fieldCount * FIELD_DESCRIPTOR_BYTES;
     if (descriptorsOff + descriptorsBytes + 4 > this._buffer.byteLength) {
@@ -114,11 +179,21 @@ export class Reader {
       const offsetInRow = this._dv.getUint16(descOff + 2, true);
       const laneKind    = this._bytes[descOff + 4];
       const flags       = this._bytes[descOff + 5];
+      const reserved2   = this._dv.getUint16(descOff + 6, true);
       const nameStrOff  = Number(this._dv.getBigUint64(descOff + 8, true));
+      const reserved3   = this._dv.getBigUint64(descOff + 16, true);
 
       if (flags !== 0) throw new ReaderError('R_BAD_FIELD_FLAGS', 'field ' + i + ' has non-zero flags (v2+ reserved)');
+      if (reserved2 !== 0) throw new ReaderError('R_RESERVED_NONZERO', 'field ' + i + ' reserved2 must be 0, got ' + reserved2);
+      if (reserved3 !== 0n) throw new ReaderError('R_RESERVED_NONZERO', 'field ' + i + ' reserved3 must be 0, got ' + reserved3);
       if (laneKind !== LANE_F64 && laneKind !== LANE_U32)
         throw new ReaderError('R_UNSUPPORTED_LANE', 'field ' + i + ' lane_kind=' + laneKind);
+      if (nameLen > 255) throw new ReaderError('R_INVALID', 'field ' + i + ' name_len ' + nameLen + ' exceeds 255');
+      if (offsetInRow + laneBytesOf(laneKind) > rowStride)
+        throw new ReaderError('R_INVALID', 'field ' + i + ' offset_in_row ' + offsetInRow +
+          ' + lane bytes exceeds row_stride ' + rowStride);
+      if (nameStrOff + nameLen > nameBlobLen)
+        throw new ReaderError('R_TRUNCATED', 'field ' + i + ' name range past the name blob');
 
       const nameBytes = this._bytes.subarray(nameBlobOff + nameStrOff, nameBlobOff + nameStrOff + nameLen);
       const name = decoder.decode(nameBytes);
@@ -133,22 +208,35 @@ export class Reader {
     const need = this._shardCount * SHARD_ENTRY_BYTES;
     if (off + need > this._buffer.byteLength) throw new ReaderError('R_TRUNCATED', 'shard directory truncated');
     const shards = new Array(this._shardCount);
+    const rowStride = this._schema.rowStride;
+    let rowSum = 0;
     for (let i = 0; i < this._shardCount; i++) {
       const entryOff = off + i * SHARD_ENTRY_BYTES;
       const payloadOff    = Number(this._dv.getBigUint64(entryOff + 0, true));
       const payloadLen    = this._dv.getUint32(entryOff + 8, true);
       const rowCount      = this._dv.getUint32(entryOff + 12, true);
       const minReaderVer  = this._dv.getUint16(entryOff + 16, true);
-      // 2 bytes flags at 18, 4 bytes reserved at 20
+      const shardFlags    = this._dv.getUint16(entryOff + 18, true);
+      const shardReserved = this._dv.getUint32(entryOff + 20, true);
       const localStrOff   = Number(this._dv.getBigUint64(entryOff + 24, true));
       const localStrLen   = Number(this._dv.getBigUint64(entryOff + 32, true));
       if (minReaderVer > READER_VERSION) {
         throw new ReaderError('R_SHARD_VERSION_TOO_NEW',
           'shard ' + i + ' requires reader version ' + minReaderVer);
       }
+      if (shardFlags !== 0) throw new ReaderError('R_RESERVED_NONZERO', 'shard ' + i + ' flags must be 0, got ' + shardFlags);
+      if (shardReserved !== 0) throw new ReaderError('R_RESERVED_NONZERO', 'shard ' + i + ' reserved must be 0, got ' + shardReserved);
+      if (payloadOff < CONTAINER_HEADER_BYTES)
+        throw new ReaderError('R_INVALID', 'shard ' + i + ' payload_off ' + payloadOff + ' overlaps the header');
       if (payloadOff + payloadLen > this._buffer.byteLength) {
         throw new ReaderError('R_TRUNCATED', 'shard ' + i + ' payload truncated');
       }
+      if (rowCount * rowStride > payloadLen)
+        throw new ReaderError('R_INVALID', 'shard ' + i + ' row_count ' + rowCount +
+          ' * row_stride ' + rowStride + ' exceeds payload_len ' + payloadLen);
+      if (localStrLen === 0 && localStrOff !== 0)
+        throw new ReaderError('R_INVALID', 'shard ' + i + ' has local_string_len 0 but local_string_off ' + localStrOff);
+      rowSum += rowCount;
       // Byte view over payload (no F64 view; row stride can be non-multiple-of-8 with U32 fields)
       const payloadBytes = new Uint8Array(this._buffer, payloadOff, payloadLen);
       const payloadDv = new DataView(this._buffer, payloadOff, payloadLen);
@@ -158,10 +246,13 @@ export class Reader {
         if (localStrOff + localStrLen > this._buffer.byteLength) {
           throw new ReaderError('R_TRUNCATED', 'shard ' + i + ' string table truncated');
         }
+        validateStringTable(this._bytes, localStrOff, localStrLen, 'shard ' + i);
         stringTable = StringTable.parse(new Uint8Array(this._buffer, localStrOff, localStrLen), 0);
       }
       shards[i] = { payloadOff, payloadLen, rowCount, payloadBytes, payloadDv, stringTable };
     }
+    if (rowSum !== this._totalRows)
+      throw new ReaderError('R_INVALID', 'shard row_count sum ' + rowSum + ' != header total_rows ' + this._totalRows);
     this._shards = shards;
   }
 
@@ -178,33 +269,38 @@ export class Reader {
   //   _zoneMapsMins/Maxes:    Float64Array  -- length shardCount * T, row-major
   _parseZoneMaps() {
     this._zoneMapsTrackedFields = null;
-    if (this._metadataOff === 0) return;
-    if (this._metadataOff + 16 > this._buffer.byteLength) return;
+    if (this._metadataOff === 0) return;   // legal absence
     const off = this._metadataOff;
-    // Check 'ZM01' magic
+    // D1: a non-zero metadata_off is a producer assertion that a parseable
+    // segment lives here. Any way it fails to parse is corruption, not absence.
+    if (off + 16 > this._buffer.byteLength)
+      throw new ReaderError('R_BAD_METADATA', 'zone maps segment header at ' + off + ' runs past the buffer');
     if (this._bytes[off]     !== 0x30 || this._bytes[off + 1] !== 0x5A ||
-        this._bytes[off + 2] !== 0x4D || this._bytes[off + 3] !== 0x31) return;
+        this._bytes[off + 2] !== 0x4D || this._bytes[off + 3] !== 0x31)
+      throw new ReaderError('R_BAD_METADATA', 'zone maps magic at ' + off + ' is not ZM01');
     const shardCount = this._dv.getUint32(off + 4, true);
     const T          = this._dv.getUint32(off + 8, true);
-    if (shardCount !== this._shardCount) {
-      throw new ReaderError('R_TRUNCATED', 'zone maps shard_count mismatch');
-    }
-    if (T === 0) return;
+    const reserved0  = this._dv.getUint32(off + 12, true);
+    if (shardCount !== this._shardCount)
+      throw new ReaderError('R_BAD_METADATA', 'zone maps shard_count ' + shardCount + ' != header ' + this._shardCount);
+    if (reserved0 !== 0)
+      throw new ReaderError('R_BAD_METADATA', 'zone maps reserved0 must be 0, got ' + reserved0);
+    if (T === 0) return;   // a well-formed segment tracking nothing: hasZoneMaps false
     const fieldTableOff = off + 16;
     const fieldTablePad = (8 - ((T * 2) & 7)) & 7;
     const minsOff = fieldTableOff + T * 2 + fieldTablePad;
     const maxesOff = minsOff + shardCount * T * 8;
     const requiredEnd = maxesOff + shardCount * T * 8;
-    if (requiredEnd > this._buffer.byteLength) {
-      throw new ReaderError('R_TRUNCATED', 'zone maps segment truncated');
-    }
+    if (requiredEnd > this._buffer.byteLength)
+      throw new ReaderError('R_BAD_METADATA', 'zone maps segment runs past the buffer');
+    if ((minsOff & 7) !== 0)
+      throw new ReaderError('R_BAD_METADATA', 'zone maps mins offset ' + minsOff + ' is not 8-byte aligned');
     const tracked = new Array(T);
     const fieldToPos = new Map();
     for (let t = 0; t < T; t++) {
       const schemaFieldIdx = this._dv.getUint16(fieldTableOff + t * 2, true);
-      if (schemaFieldIdx >= this._schema.fields.length) {
-        throw new ReaderError('R_TRUNCATED', 'zone maps references field index ' + schemaFieldIdx);
-      }
+      if (schemaFieldIdx >= this._schema.fields.length)
+        throw new ReaderError('R_BAD_METADATA', 'zone maps references field index ' + schemaFieldIdx + ' past the schema');
       tracked[t] = schemaFieldIdx;
       fieldToPos.set(this._schema.fields[schemaFieldIdx].name, t);
     }
@@ -259,6 +355,8 @@ export class Reader {
   //   - the field is not tracked (currently: U32/string fields are never tracked in v1).
   // For empty ranges (a shard where the field defaulted to 0), returns {min:0, max:0}.
   shardBounds(shardIdx, fieldName) {
+    if (!this._fieldIndex.has(fieldName))
+      throw new ReaderError('R_UNKNOWN_FIELD', 'no field named ' + JSON.stringify(fieldName));
     if (!this._zoneMapsTrackedFields) return null;
     const t = this._zoneMapsFieldToPos.get(fieldName);
     if (t === undefined) return null;
@@ -273,6 +371,8 @@ export class Reader {
   // Semantics: a shard is INCLUDED iff its [smin, smax] overlaps [min, max].
   // A pass-through range ({min: -Inf, max: +Inf} or omitted) returns all shards.
   findShards(fieldName, opts) {
+    if (!this._fieldIndex.has(fieldName))
+      throw new ReaderError('R_UNKNOWN_FIELD', 'no field named ' + JSON.stringify(fieldName));
     const all = () => { const a = new Array(this._shardCount); for (let i = 0; i < this._shardCount; i++) a[i] = i; return a; };
     if (!this._zoneMapsTrackedFields) return all();
     const t = this._zoneMapsFieldToPos.get(fieldName);

package/src/Split.js

@@ -37,7 +37,7 @@ import { Reader, ReaderError } from './Reader.js';
 import { StringTable } from './StringTable.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 const LF = 0x0A;
 const CONTAINER_HEADER_BYTES = 48;
@@ -45,6 +45,7 @@ const SHARD_ENTRY_BYTES = 40;
 const FIELD_DESCRIPTOR_BYTES = 24;
 const FOOTER_BYTES = 16;
 const LANE_F64 = 1;
+const LANE_U32 = 3;
 const U32_MAX = 4294967295;
 
 const SPLIT_NDJSON_OPTS = {
@@ -201,6 +202,16 @@ export function mergeContainers(containers) {
     }
   }
 
+  // BS-15 conformance: a merged container whose schema has no U32 lane must
+  // emit NO local string tables (local_string_off/len = 0), regardless of what
+  // the source containers carry. A pre-M2 F64-only input holds a stale 16-byte
+  // empty table; without this, merge would launder it into the output and the
+  // t8 spec-checker would reject the result. Computed once from the shared schema.
+  let mergedHasU32 = false;
+  for (let i = 0; i < schema0.fields.length; i++) {
+    if (schema0.fields[i].laneKind === LANE_U32) { mergedHasU32 = true; break; }
+  }
+
   // Total shards and rows
   let totalShards = 0, totalRows = 0;
   for (const r of readers) { totalShards += r.shardCount; totalRows += r.totalRows; }
@@ -246,7 +257,7 @@ export function mergeContainers(containers) {
       // Source shard payload lives at shard.payloadOff in the source container.
       const payloadLen = shard.payloadLen;
       const payloadPad = (8 - (payloadLen & 7)) & 7;
-      const stringLen = shard.stringTable ? _sourceStringTableLen(readers[ci], si) : 0;
+      const stringLen = (mergedHasU32 && shard.stringTable) ? _sourceStringTableLen(readers[ci], si) : 0;
       const outPayloadOff = cursor;
       const outStringOff = outPayloadOff + payloadLen + payloadPad;
       shardOutMap.push({

package/src/StringTable.js

@@ -20,7 +20,7 @@
 // and at every reset(), so an absent U32 row cell (which is 0) decodes as ""
 // rather than aliasing the shard's first-interned string (SPEC 3.3, SPEC 7).
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 const EMPTY_SLOT = 0xFFFFFFFF;   // MUST be unsigned; typed-array reads are unsigned
 const INITIAL_BLOB_BYTES  = 64 * 1024;

package/src/Tokenizer.js

@@ -10,7 +10,6 @@
 //   E_INVALID_ESCAPE       - unknown \ sequence
 //   E_INVALID_HEX          - non-hex digit in \uXXXX
 //   E_UNPAIRED_SURROGATE   - lone high/low surrogate in \u escape
-//   E_INVALID_UTF8         - malformed UTF-8 byte sequence
 //   E_NUMBER_OVERFLOW      - number magnitude exceeds F64 range
 //   E_NUMBER_INVALID       - malformed number literal
 //   E_KEYWORD_MISMATCH     - true/false/null spelled wrong
@@ -24,7 +23,7 @@
 
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 const U32_MAX = 4294967295;
 const TOKENIZER_OPTS = {
@@ -181,8 +180,12 @@ export class Tokenizer {
 
   // Feed a chunk. Bytes are consumed synchronously; sink events fire during this call.
   feed(chunk) {
-    if (this._poisoned) throw new TokenizerError('E_POISONED', this._absOffset, 'tokenizer poisoned by a previous error; construct a new instance');
+    if (this._poisoned) throw new TokenizerError('E_POISONED', this._absOffset, 'tokenizer poisoned by a previous error or an in-flight feed(); construct a new instance');
     if (this._ended) throw new TokenizerError('E_ENDED', this._absOffset, 'tokenizer already ended; construct a new instance');
+    // D3 armed-flag: a sink throw mid-feed unwinds through here with the flag
+    // set, poisoning the instance. Cleared on the successful fall-through below.
+    // Two boolean stores per chunk; the byte loop is untouched.
+    this._poisoned = true;
     const len = chunk.length;
     let i = 0;
     while (i < len) {
@@ -342,11 +345,13 @@ export class Tokenizer {
       // unreachable
       this._err('E_UNEXPECTED_BYTE', 'internal: unknown state ' + st);
     }
+    this._poisoned = false;   // D3: clean fall-through un-arms the flag
   }
 
   end() {
-    if (this._poisoned) throw new TokenizerError('E_POISONED', this._absOffset, 'tokenizer poisoned by a previous error; construct a new instance');
+    if (this._poisoned) throw new TokenizerError('E_POISONED', this._absOffset, 'tokenizer poisoned by a previous error or an in-flight feed(); construct a new instance');
     if (this._ended) throw new TokenizerError('E_ENDED', this._absOffset, 'tokenizer already ended; construct a new instance');
+    this._poisoned = true;    // D3: armed until a clean exit below
     // A trailing number may still be pending
     if (this._state === S_NUMBER) {
       this._emitNumber();
@@ -357,6 +362,7 @@ export class Tokenizer {
       }
       this._ended = true;
       this._onEnd();
+      this._poisoned = false;   // clear AFTER _onEnd(): a throwing onEnd stays poisoned
       return;
     }
     this._err('E_UNEXPECTED_EOF', 'input ended mid-token (state=' + this._state + ')');

package/src/Views.js

@@ -0,0 +1,25 @@
+// @zakkster/lite-bake-stream / Views (internal)
+// Copyright (c) 2026 Zahary Shinikchiev. MIT.
+//
+// One shared helper so every container entry point (deserialize,
+// Reader.fromBuffer, PreserveReader.fromBuffer) resolves caller-supplied bytes
+// to an ArrayBuffer by exactly the SAME rule (BS-09). A Uint8Array that is a
+// full-buffer view is unwrapped in place (zero copy); a partial view (a pooled
+// Buffer with byteOffset > 0, or a subarray) is COPIED so the reader never sees
+// bytes outside the caller's window. Anything else fails closed.
+//
+// Not a public export. Internal to the package.
+
+export function toContainerBuffer(input, label) {
+  if (input instanceof ArrayBuffer) return input;
+  if (input instanceof Uint8Array) {
+    if (input.byteOffset === 0 && input.byteLength === input.buffer.byteLength) {
+      return input.buffer;
+    }
+    const copy = new Uint8Array(input.byteLength);
+    copy.set(input);
+    return copy.buffer;
+  }
+  throw new TypeError(label + ': expected ArrayBuffer or Uint8Array, got ' +
+    (input === null ? 'null' : typeof input));
+}

package/src/Writer.js

@@ -36,7 +36,7 @@
 import { StringTable } from './StringTable.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 const U32_MAX = 4294967295;
 // Post-finalize sentinel for _recordDepth. Chosen = 2 so every post-finalize
@@ -64,6 +64,11 @@ const FIELD_FLAGS_NONE = 0;
 const SHARD_MIN_READER_VERSION = 1;
 const SHARD_FLAGS_NONE = 0;
 
+// Shared empty view for F64-only shards, which emit NO local string table
+// (SPEC 3.4: local_string_off/local_string_len are 0 when a shard has no
+// U32-lane fields). Read-only; never written into.
+const EMPTY_STRING_TABLE_BYTES = new Uint8Array(0);
+
 const FORMAT_VERSION = 1;
 const ENDIAN_LE = 1;
 
@@ -359,6 +364,12 @@ export class Writer {
 
   _finalizeSchema(fields, rowStride) {
     this._schema = { fields, rowStride };
+    // BS-15: a shard with no U32-lane field emits NO local string table.
+    // Computed once at freeze, consulted per shard at finalize / assembly.
+    this._hasU32 = false;
+    for (let i = 0; i < fields.length; i++) {
+      if (fields[i].laneKind === LANE_U32) { this._hasU32 = true; break; }
+    }
 
     const enc = new TextEncoder();
     this._fieldNamesUtf8 = new Array(fields.length);
@@ -492,9 +503,11 @@ export class Writer {
     const usedBytes = this._currentShardRowCount * this._schema.rowStride;
     const copy = new Uint8Array(usedBytes);
     copy.set(this._currentShardBytes.subarray(0, usedBytes));
-    // Serialize the current shard's string table (empty for F64-only shards).
+    // Serialize the current shard's string table. A shard whose schema has no
+    // U32 lane emits NO table at all (BS-15 / SPEC 3.4): zero bytes here means
+    // the ShardEntry carries local_string_off = 0 / local_string_len = 0.
     const st = this._perShardStringTable || this._stringTable;
-    const stSer = st.serialize();
+    const stBytes = this._hasU32 ? st.serialize().bytes : EMPTY_STRING_TABLE_BYTES;
     // Snapshot zone-map bounds for this shard. Copy so subsequent shards can
     // reuse _currentShardMins/Maxes without clobbering the pushed record.
     const T = this._trackedFieldIndices.length;
@@ -505,7 +518,7 @@ export class Writer {
     this._shards.push({
       bytes: copy,
       rowCount: this._currentShardRowCount,
-      stringTableBytes: stSer.bytes,
+      stringTableBytes: stBytes,
       mins: shardMins,
       maxes: shardMaxes,
     });
@@ -721,7 +734,7 @@ export class Writer {
       dv.setUint16(entryOff + 16, SHARD_MIN_READER_VERSION, true);
       dv.setUint16(entryOff + 18, SHARD_FLAGS_NONE, true);
       dv.setUint32(entryOff + 20, 0, true);   // reserved
-      dv.setBigUint64(entryOff + 24, BigInt(shardStringTableOffsets[i]), true);
+      dv.setBigUint64(entryOff + 24, BigInt(shardStringTableLens[i] > 0 ? shardStringTableOffsets[i] : 0), true);
       dv.setBigUint64(entryOff + 32, BigInt(shardStringTableLens[i]), true);
     }
 

package/src/index.js

@@ -17,6 +17,7 @@ import { PreserveTokenizer, PreserveTokenizerError } from './PreserveTokenizer.j
 import { PreserveWriter, PreserveWriterError } from './PreserveWriter.js';
 import { PreserveReader, PreserveReaderError } from './PreserveReader.js';
 import { checkOpts } from './Opts.js';
+import { toContainerBuffer } from './Views.js';
 
 export {
   Tokenizer, TokenizerError,
@@ -27,7 +28,7 @@ export {
   PreserveWriter, PreserveWriterError,
   PreserveReader, PreserveReaderError,
 };
-export const VERSION = '1.2.0';
+export const VERSION = '1.3.1';
 
 const encoder = new TextEncoder();
 
@@ -188,20 +189,7 @@ async function _serializeAsyncIterablePreserve(iterable, opts) {
 // Deserialize LBK1 bytes into the right Reader for the container's mode.
 // Auto-detects preserve vs schema via header flag byte at offset 7 bit 0.
 export function deserialize(bytes) {
-  let buffer;
-  if (bytes instanceof ArrayBuffer) {
-    buffer = bytes;
-  } else if (bytes instanceof Uint8Array) {
-    if (bytes.byteOffset === 0 && bytes.byteLength === bytes.buffer.byteLength) {
-      buffer = bytes.buffer;
-    } else {
-      const copy = new Uint8Array(bytes.byteLength);
-      copy.set(bytes);
-      buffer = copy.buffer;
-    }
-  } else {
-    throw new TypeError('deserialize: expected Uint8Array or ArrayBuffer, got ' + typeof bytes);
-  }
+  const buffer = toContainerBuffer(bytes, 'deserialize');
   if (buffer.byteLength < 8) {
     throw new ReaderError('R_TRUNCATED', 'container too small to inspect header flags');
   }