@zakkster/lite-bake-stream 1.4.0 → 1.4.1

package/CHANGELOG.md

@@ -2,6 +2,29 @@
 
 All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.4.1] -- 2026-09-02
+
+M8 -- the four reserved torture cells become real; BS-30 and BS-31 close. No tier prints "reserved" any more. Suite: 461 tests / 461 pass / 0 fail / 0 todo (unchanged -- no named-suite edits). Torture: 44/44 fast (~1.3 s wall), 47/47 full, arrayBuffers growth 0. BREAK matrix: ten runs (`=1`, eight registry ids, one unknown id) all exit non-zero; the unknown id is refused at import. Inventory gate: 54 thrown codes / 0 unpinned. Falsifiability: with baseline src stashed, exactly one new assertion fails (the onProgress-identity pin), both tiers. Test-only except one hot-path reuse in `src/FileIngest.js` (below).
+
+### Changed
+
+- BS-31 (hot path): `ingestStream`'s per-chunk `onProgress` state object is now allocated ONCE and mutated in place, removing a per-chunk object literal from the ingest read loop. The state object is a reused ephemeral view -- valid only during the callback; copy it if retained. Contract documented in `src/FileIngest.js`, `types/FileIngest.d.ts`, and `llms.txt`. Falsifiability: with baseline src stashed, exactly the new onProgress-identity pin fails (baseline allocates a fresh object per chunk); every other new cell passes on both trees.
+- Torture cells t0/t1/t2/t4 filled in. t0: the re-chunk determinism law (byte-identical container across whole/string/1/7/64/4096-byte chunkings x both modes on a shared degenerate corpus) plus mergeContainers/compileInParts/MultiReader query-equivalence, findShards superset+point-exact, and splitNDJSON reassembly. t1: degenerate-value and refusal pins over the shared corpus (`test/torture/fixtures/degenerate.mjs`, reused by t0/t2 and MQ2). t2: every-byte-position split-invariance fuzz over both tokenizers. t4: API-abuse decided-policy pins plus the R5 adversarial doors.
+- BREAK selector: `BAKE_TORTURE_BREAK` now accepts a registry id (t6, g, i, j, k, l, m, n) to arm exactly one break site -- each control proves its own exit-non-zero path -- alongside legacy `=1` (arm all, halt at first). An unknown value fails closed at import. Registry documented in `test/torture/harness.mjs`.
+- Demo (repo-only, never shipped in the tarball): refreshed from its pre-1.0 state to the current release. The synthetic fixture now exercises the v1.4 inference-window semantics (~8% null string cells read back `""` with the lane still U32; ~8% absent numeric cells read `0`); schema-mode ingest uses `framing: 'auto'` so a dropped top-level `[...]` array works; error handling matches on stable codes (`W_NESTED_UNSUPPORTED`) instead of message text; the onProgress reuse contract is stated where the demo destructures the state object; both demo files are ASCII-clean. Every library call the demo makes was verified headless against `src/` (24 checks green, incl. both handled error paths).
+
+### Added
+
+- BS-30: the code-inventory gate (`test/torture/inventory.mjs`, run from t8). It scans `src/*.js` for every thrown stable code (constructed-error and `_err`/`raise*` families), asserts each is pinned by at least one test, and prints its collected-code count (54). The six previously-open doors are pinned in t3/t4 (`R_UNSUPPORTED_VERSION`, `R_UNSUPPORTED_ENDIAN`, `R_ADAPTER_SHORT_READ`, `W_ROW_STRIDE_ZERO` x2 sites, `E_RECORD_TOO_LARGE`, `E_UNSUPPORTED_FRAMING`); t1 pins the two remaining unpinned degenerate codes (`E_INVALID_HEX`, `E_TRAILING_INPUT`). A t9 control feeds the scanner a synthetic unpinned code to prove the gate can flag one.
+- BS-31: a t7 mid-stream-failure witness -- a ReadableStream whose pull throws makes `ingestStream` reject with the source error (no half-built Reader), and the lite-leak tracker returns to size 0. t6 gates the ingest sync core (feed + reused-progress mutation + `onProgress`) at zero allocation; the async await-read loop is covered by the t7 witness (runOpsGate rejects async fns).
+- t9 controls (i)-(n): one in-run rehearsal per newly-real cell plus the inventory gate, so a plain `npm run torture` proves every new gate can fail.
+
+### Findings (ledgered, no ride-along fix per session non-goals)
+
+- `Reader.get(1.5)` returns garbage (a fractional-byte-offset read) instead of a decided policy (throw / no-op / undefined) -- t4 records it, does not pin it as correct; the t9 (l) control proves the gate would catch a garbage return.
+- `RangeReader.syncRange` without a prior `prefetchRange` throws the shared `R_TRUNCATED`; the ROADMAP calls for a distinct code. t4 records it, does not pin the shared code.
+- `RangeReader` never length-validates an adapter's `fetch()` return: an adapter that returns fewer bytes than requested surfaces a raw downstream `RangeError`, not `R_ADAPTER_SHORT_READ` (whose two real sites are the Mock adapter's own past-size check and `HTTPRangeAdapter`'s 200-not-206 check). Ledgered as a reader-guard gap.
+
 ## [1.4.0] -- 2026-09-01
 
 M4 -- the sample window does what its name says, and null stops eating lanes. Two inference-window bugs fixed together (BS-07, BS-20). Suite: 461 tests / 461 pass / 0 fail / 0 todo. Torture: 44/44 fast, 47/47 full (incl. the 64 MB structural scenario), arrayBuffers growth 0. Mutation measurement: the 8-test manifest failed exactly and the 5 canaries passed on baseline src. Findings probes: 19 NOT-REPRODUCED / 0 REPRODUCED / 0 PARTIAL -- the ledger's first all-clear; the probe file is retired.

package/README.md

@@ -13,7 +13,7 @@
 
 > Streaming byte-level JSON compiler for [`@zakkster/lite-bake`](https://github.com/PeshoVurtoleta/lite-bake). Zero-GC, tree-shakeable, gigabyte-scale.
 
-**Status:** v1.4.0. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak with zero GC and byte-exact preservation across 590 million cells.
+**Status:** v1.4.1. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak with zero GC and byte-exact preservation across 590 million cells.
 
 ## Two modes, one API
 

package/llms.txt

@@ -8,13 +8,13 @@ Ingest gigabyte-scale JSON (top-level array or NDJSON) into the `lite-bake` LBK1
 
 ## Status
 
-v1.4.0 — stable. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak (M1 MacBook Pro): 98.37M rows, 4.89 GB container, zero major GC, zero minor GC, 499 KB total heap allocation, 590.21M cells verified byte-exact, zero mismatches. Tokenizer benches at 222-237 MB/s (~55% of JSON.parse, with no object graph allocated).
+v1.4.1 — stable. LBK1 format frozen at `format_version: 1`. Qualified on an 8 GB soak (M1 MacBook Pro): 98.37M rows, 4.89 GB container, zero major GC, zero minor GC, 499 KB total heap allocation, 590.21M cells verified byte-exact, zero mismatches. Tokenizer benches at 222-237 MB/s (~55% of JSON.parse, with no object graph allocated).
 
 Public API follows semver from 1.0.0. Future additions (I64 lane, columnar payload mode, container-level string table) land via the format's forward-compat seams -- `min_reader_version` on ShardEntry, reserved FieldDescriptor flags, the `metadata_off` block wrapper -- without a format_version bump.
 
 See SPEC.md for the LBK1 container format, section 3.6 for zone maps, section 4.3 for the reserved field flags.
 
-## Public API (v1.4.0)
+## Public API (v1.4.1)
 
 Two ingest modes share one top-level API:
 
@@ -41,7 +41,7 @@ Shared:
 - `ingestStream`, `ingestFile` (`/file-ingest`): browser helpers piping a `ReadableStream<Uint8Array>` through the pipeline. `preserve` option dispatches to the right writer.
 
 Error classes with stable `code`: `TokenizerError`, `WriterError`, `ReaderError`, `RangeReaderError`, `MultiReaderError`, `SplitError`, `PreserveTokenizerError`, `PreserveWriterError`, `PreserveReaderError`.
-- `ingestStream`, `ingestFile` (`/file-ingest`): browser helpers piping a `ReadableStream<Uint8Array>` (e.g. `File.stream()`) through the Tokenizer + Writer, returning a Reader. Per-chunk `onProgress` callback.
+- `ingestStream`, `ingestFile` (`/file-ingest`): browser helpers piping a `ReadableStream<Uint8Array>` (e.g. `File.stream()`) through the Tokenizer + Writer, returning a Reader. Per-chunk `onProgress` callback; its state object is reused across calls (mutated in place, zero per-chunk allocation) -- copy it if retained past the callback.
 - `TokenizerError`, `WriterError`, `ReaderError`, `RangeReaderError`: thrown on parse/write/read errors with stable `code`.
 - `VERSION` const per subpath.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@zakkster/lite-bake-stream",
-  "version": "1.4.0",
+  "version": "1.4.1",
   "description": "Streaming byte-level JSON to lite-bake binary compiler. Zero-GC, tree-shakeable, gigabyte-scale.",
   "type": "module",
   "sideEffects": false,

package/src/FileIngest.js

@@ -27,7 +27,7 @@ import { PreserveWriter } from './PreserveWriter.js';
 import { PreserveReader } from './PreserveReader.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 const U32_MAX = 4294967295;
 const INGEST_OPTS = {
@@ -49,7 +49,12 @@ function raiseIngest(code, msg) { throw new WriterError(code, msg); }
 //                       sampleBytes }. Preserve: { targetShardBytes, maxRecordBytes }.
 //   onProgress(state):  called after each chunk with { bytesIngested,
 //                       totalBytes, rowsWritten, shardsCommitted, chunkBytes,
-//                       chunkCount, elapsedMs }
+//                       chunkCount, elapsedMs }. The state object is REUSED
+//                       across calls -- one object is mutated in place per chunk
+//                       (zero per-chunk allocation), so it is valid only during
+//                       the callback; copy it if you retain it past the call.
+//                       Same ephemeral-view contract as the tokenizer's byte
+//                       ranges.
 //   totalBytes:         optional total for progress percentage. Pass file.size
 //                       when ingesting a File; omit for indeterminate streams.
 export async function ingestStream(readableStream, opts) {
@@ -73,10 +78,20 @@ export async function ingestStream(readableStream, opts) {
     : new Tokenizer(writer, { framing });
 
   const reader = readableStream.getReader();
-  const t0 = (globalThis.performance && performance.now) ? performance.now() : Date.now();
+  const nowFn = (globalThis.performance && performance.now)
+    ? () => performance.now() : () => Date.now();
+  const t0 = nowFn();
   let bytesIngested = 0;
   let chunkCount = 0;
 
+  // One progress object, hoisted and mutated per chunk -- the ingest loop
+  // allocates nothing per chunk. Documented as a reused ephemeral view above.
+  const progress = onProgress ? {
+    bytesIngested: 0, totalBytes,
+    rowsWritten: 0, shardsCommitted: 0,
+    chunkBytes: 0, chunkCount: 0, elapsedMs: 0,
+  } : null;
+
   try {
     while (true) {
       const { value, done } = await reader.read();
@@ -87,16 +102,14 @@ export async function ingestStream(readableStream, opts) {
       tokenizer.feed(value);
       bytesIngested += value.length;
       chunkCount++;
-      if (onProgress) {
-        const now = (globalThis.performance && performance.now) ? performance.now() : Date.now();
-        onProgress({
-          bytesIngested, totalBytes,
-          rowsWritten: writer.totalRows,
-          shardsCommitted: writer.shardCount,
-          chunkBytes: value.length,
-          chunkCount,
-          elapsedMs: now - t0,
-        });
+      if (progress) {
+        progress.bytesIngested = bytesIngested;
+        progress.rowsWritten = writer.totalRows;
+        progress.shardsCommitted = writer.shardCount;
+        progress.chunkBytes = value.length;
+        progress.chunkCount = chunkCount;
+        progress.elapsedMs = nowFn() - t0;
+        onProgress(progress);
       }
     }
   } finally {

package/src/MultiReader.js

@@ -25,7 +25,7 @@
 //   M_ROW_OUT_OF_RANGE    - rowIdx >= totalRows
 //   M_TOO_MANY_ROWS       - cumulative row count exceeds Number.MAX_SAFE_INTEGER
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 export class MultiReaderError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'MultiReaderError'; }

package/src/PreserveReader.js

@@ -32,7 +32,7 @@
 
 import { toContainerBuffer } from './Views.js';
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 const CONTAINER_HEADER_BYTES = 48;
 const SHARD_ENTRY_BYTES = 40;

package/src/PreserveTokenizer.js

@@ -32,7 +32,7 @@
 
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 const U32_MAX = 4294967295;
 const PRESERVE_TOKENIZER_OPTS = {

package/src/PreserveWriter.js

@@ -18,7 +18,7 @@
 
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 export class PreserveWriterError extends Error {
   constructor(code, msg) { super(msg); this.code = code; this.name = 'PreserveWriterError'; }

package/src/RangeReader.js

@@ -42,7 +42,7 @@
 import { StringTable } from './StringTable.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 const CONTAINER_HEADER_BYTES = 48;
 const SHARD_ENTRY_BYTES = 40;

package/src/Reader.js

@@ -25,7 +25,7 @@
 import { StringTable } from './StringTable.js';
 import { toContainerBuffer } from './Views.js';
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 const CONTAINER_HEADER_BYTES = 48;
 const SHARD_ENTRY_BYTES = 40;

package/src/Split.js

@@ -37,7 +37,7 @@ import { Reader, ReaderError } from './Reader.js';
 import { StringTable } from './StringTable.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 const LF = 0x0A;
 const CONTAINER_HEADER_BYTES = 48;

package/src/StringTable.js

@@ -20,7 +20,7 @@
 // and at every reset(), so an absent U32 row cell (which is 0) decodes as ""
 // rather than aliasing the shard's first-interned string (SPEC 3.3, SPEC 7).
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 const EMPTY_SLOT = 0xFFFFFFFF;   // MUST be unsigned; typed-array reads are unsigned
 const INITIAL_BLOB_BYTES  = 64 * 1024;

package/src/Tokenizer.js

@@ -30,7 +30,7 @@
 
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 const U32_MAX = 4294967295;
 const TOKENIZER_OPTS = {

package/src/Writer.js

@@ -43,7 +43,7 @@
 import { StringTable } from './StringTable.js';
 import { checkOpts } from './Opts.js';
 
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 const U32_MAX = 4294967295;
 // Post-finalize sentinel for _recordDepth. Chosen = 2 so every post-finalize

package/src/index.js

@@ -28,7 +28,7 @@ export {
   PreserveWriter, PreserveWriterError,
   PreserveReader, PreserveReaderError,
 };
-export const VERSION = '1.4.0';
+export const VERSION = '1.4.1';
 
 const encoder = new TextEncoder();
 

package/types/FileIngest.d.ts

@@ -6,6 +6,11 @@ import type { WriterOptions } from './Writer.d.ts';
 
 export const VERSION: string;
 
+/**
+ * Progress state passed to onProgress. The SAME object is reused across every
+ * chunk callback (mutated in place, zero per-chunk allocation): it is valid only
+ * during the callback -- copy it if you retain it past the call.
+ */
 export interface IngestProgress {
   bytesIngested: number;
   totalBytes: number;