videobook-engine 4.2.0 → 5.3.2

package/docs/doltlite-staging.md

@@ -0,0 +1,69 @@
+# DoltLite staging and native merge verification
+
+The engine pins DoltLite 0.50.6. It includes the fix for
+[dolthub/doltlite#2644](https://github.com/dolthub/doltlite/issues/2644),
+merged in [PR 2646](https://github.com/dolthub/doltlite/pull/2646) and
+published in [0.50.6](https://github.com/dolthub/doltlite/releases/tag/v0.50.6).
+The dependency adoption is tracked in `ve-ovz.23`; the remaining native-merge
+failure stays in `ve-wsu`.
+
+## Fixed: incremental staging corrupts index roots
+
+Staging twelve newly created tables individually produced invalid committed
+UNIQUE-index roots on 0.11.37 and 0.50.5. Indexed reads worked before checkout,
+then failed after checking out a branch at the same commit. Both controls now
+pass on 0.50.6:
+
+```sh
+node scripts/dolt-staging-probe.cjs @dolthub/doltlite
+node scripts/dolt-staging-probe.cjs @dolthub/doltlite --stage-all
+```
+
+Full engine URL bootstrap, backed-up catalog cloning, lazy object reads,
+post-clone writes and reopen are verified in `tests/fork-flow.test.ts`.
+`tests/merge-policy.test.ts` verifies repeated full-catalog checkout preserves
+all 56 engine tables, ignored runtime rows, indexed file lookups, and integrity.
+The transcript and primary-sequence tests now perform both native merges
+instead of inserting the second branch's expected rows directly. Native row
+conflicts are mapped to `MERGE_CONFLICT` and preserve the accepted head.
+
+The upstream change prevents new corrupt commits; it does not rewrite
+previously corrupted history. Snapshot bootstrap remains available for a
+healthy working catalog whose old committed schema cannot be cloned.
+
+## Remaining: ignored runtime tables block native merge
+
+A separate failure reproduces on 0.50.6 with one versioned table and one
+ignored runtime table. Native merge refuses with an uncommitted-changes error
+even after checkout and hard reset. Without an index, `dolt_status` is empty;
+with a secondary runtime index it also incorrectly reports the ignored table
+as modified. The control without the runtime table passes:
+
+```sh
+node scripts/dolt-ignored-merge-probe.cjs @dolthub/doltlite --without-runtime
+node scripts/dolt-ignored-merge-probe.cjs @dolthub/doltlite --without-index
+node scripts/dolt-ignored-merge-probe.cjs @dolthub/doltlite
+```
+
+The latter two commands deliberately exit nonzero when the defect reproduces.
+All probes use temporary synthetic catalogs and remove them on exit. Replace
+the package argument with an absolute installed-package directory to test a
+future dependency version. The engine retains its working projection merge,
+including singleton reconciliation and forget-wins object handling, while
+this native merge gate remains unresolved.
+
+A [native source patch with validation and reproduction steps](../patches/doltlite/README.md)
+is now prepared on upstream commit `37a390eb7b021962d9d287a465a2da3c9f59c3cf`.
+It passes 4,156 focused checks including allocation failures, all 126 native
+suites, all 33 C suites, and the full 56-table engine catalog probe. Production
+adoption still requires upstream review and validation of a published native
+build; the dependency remains 0.50.6.
+
+Compatibility smoke tests created separate synthetic catalogs with the previous
+engine dependency (0.11.37) and the installed application dependency (0.11.51),
+then opened them in separate 0.50.6 processes. Book identity, head, history
+count, artifact/notebook projections, every table row count, source bytes, and
+the runtime setting were preserved; historical restore, a new write, and
+reopen passed. The engine fixture also retained the same semantic projections.
+Dolt status diagnostics changed between versions and were compared separately
+from stored state. These tests do not claim to repair old corrupt commits.

package/docs/edit-performance.md

@@ -0,0 +1,83 @@
+# Edit preview and commit distributions
+
+VE-NFR-005, VE-NFR-006 and VE-NFR-009 require a 100-operation batch against a
+1,000-clip sequence to preview in under 250 ms p95 without storage mutation,
+commit in under 1 second p95 on a healthy local catalog, and preview
+deterministically. `tests/edit-transactions.test.ts` still measures one
+preview and one commit as a coarse regression tripwire; distributions come
+from the dedicated harness:
+
+```bash
+npm run benchmark:edits -- --output /tmp/edit-distribution.json --assert
+```
+
+The harness seeds one video stream and inserts `--clips` one-frame clips
+(default 1,000) through a single edit commit. It then runs `--commits`
+independent transactions (default 50). Every transaction:
+
+1. reads the primary sequence at the current head and builds a fresh
+   `--operations`-operation batch (default 100 `set-clip-transform`
+   operations on a rotating window of distinct clips, with transform values
+   that differ per transaction so each commit changes every addressed clip);
+2. records the head revision, every table row count and the canonical
+   sequence projection, previews the batch (timed as `preview`), and requires
+   all three snapshots to be unchanged afterwards;
+3. previews the identical intent again (timed as `preview.repeat`) and
+   requires identical canonical operations, affected ranges, write set,
+   diff, before/after hashes and preview hash;
+4. commits with the preview hash (timed as `commit`), requires the head to
+   advance to the returned revision, and requires the committed sequence to
+   carry every transform.
+
+After the last commit the catalog is closed and reopened, and the reopened
+sequence must expose the final revision and clip count. The report records
+nearest-rank p50/p95/max and every raw sample for `preview`,
+`preview.repeat`, `commit` and the seeding operations, the invariant counters,
+process peak RSS, Node/CPU/memory, source commit and dirty state, and the
+gates: `fullScale` (1,000 clips, 100 operations, 50 commits),
+`previewP95Under250Ms`, `commitP95Under1Second`, `noStorageMutation`,
+`deterministicPreviews` and `everyCommitAdvancedRevision`. `--assert` exits
+nonzero when any gate fails. `tests/edit-benchmark.test.ts` runs a small
+workload and checks the report structure and invariants on every test run.
+
+Commit timings exclude derived jobs, which the engine does not start from
+`edits.commit`. The workload uses transform operations only; other operation
+kinds share the same projection, validation and persistence path but are not
+separately measured here.
+
+## Recorded run
+
+The [recorded distribution](../benchmarks/results/edit-100x1000-distribution.json)
+measured the working tree on top of engine `8b235f2` on September 6, 2026,
+using an Apple M1 Pro with 16 GB RAM and Node 24.10.0 while no other benchmark
+was running:
+
+| Measurement | Samples | p50 | p95 | Max |
+| --- | ---: | ---: | ---: | ---: |
+| Preview (100 operations, 1,000 clips) | 50 | 106 ms | 110 ms | 113 ms |
+| Repeated preview of the same intent | 50 | 106 ms | 110 ms | 112 ms |
+| Commit | 50 | 300 ms | 334 ms | 335 ms |
+
+Seeding the 1,000 clips took 142 ms to preview and 297 ms to commit. All 50
+previews left the head revision, row counts and sequence projection unchanged,
+all 50 repeats were identical, all 50 commits advanced the head and applied
+5,000 transforms, and every gate passed. Process peak RSS was 2.8 GiB
+including the embedded Dolt database. The PRD reference device is an M2 Pro
+with 16 GB RAM; this run does not replace a reference-device measurement.
+
+## DoltLite 0.50.6 candidate
+
+Engine `c4f1d89` (5.3.2) was measured from a clean committed tree on the same
+M1 Pro, 16 GB RAM, Node 24.10.0. The
+[complete report](../benchmarks/results/edit-100x1000-doltlite-0506.json)
+records 50 independent 100-operation transactions over 1,000 clips:
+
+| Measurement | Samples | p50 | p95 | Max |
+| --- | ---: | ---: | ---: | ---: |
+| Preview | 50 | 105 ms | 114 ms | 118 ms |
+| Repeated preview | 50 | 105 ms | 111 ms | 115 ms |
+| Commit | 50 | 299 ms | 331 ms | 348 ms |
+
+All mutation, determinism, revision, and applied-transform gates pass. Peak
+process RSS is 1.54 GiB. These M1 Pro results retain the M2 Pro reference-device
+qualification; `ve-ovz.22` remains outstanding.

package/docs/engine-layout.md

@@ -79,6 +79,12 @@ flowchart LR
 - JSON is stored as canonical text with recursively sorted object keys.
 - Deletes are hard deletes. Owned rows cascade; live artifact/entity
   references restrict deletion; prior Dolt revisions remain available.
+  `artifacts.delete(id, { deleteOwnedMedia: true })` explicitly removes the
+  artifact's streams and transcripts in the same semantic transaction.
+  Sequence clips, caption selections, notebook references, and pinned search
+  results still block deletion. Omitting the option retains the default
+  stream/transcript reference guard. Temporal runtime rows are removed with
+  the artifact; prior source and transcript revisions remain readable.
 - Objects are content-immutable but forgettable. `engine.storage.deleteObject`
   forgets one object (refusing `IN_USE` references at HEAD unless forced);
   `engine.storage.gc` sweeps every object nothing references at HEAD.
@@ -106,8 +112,8 @@ There are 34 allowlisted semantic tables.
 | Table | Columns | Keys, constraints, and purpose |
 | --- | --- | --- |
 | `engine_schema` | `singleton INTEGER`<br>`version INTEGER`<br>`created_at INTEGER` | `singleton PK CHECK(singleton = 1)`. Records the clean-break catalog version. The recorded version rejects older catalogs rather than migrating them. |
-| `book` | `book_id TEXT`<br>`slug TEXT`<br>`created_at INTEGER` | `book_id PK`; `slug UQ`. Exactly one row per engine root. |
-| `artifacts` | `artifact_id TEXT`<br>`slug TEXT`<br>`kind TEXT`<br>`created_at INTEGER` | `artifact_id PK`; `slug UQ`; `kind CHECK IN (video, image, audio, script, character, prompt, scene, final)`. The artifact is the stable identity for source media, generated media, documents, and final renders. |
+| `book` | `book_id TEXT`<br>`name TEXT`<br>`created_at INTEGER` | `book_id PK`. Exactly one row per engine root; `name` is free-text display. |
+| `artifacts` | `artifact_id TEXT`<br>`label TEXT?`<br>`kind TEXT`<br>`created_at INTEGER` | `artifact_id PK`; `kind CHECK IN (video, image, audio, script, character, prompt, scene, final)`. `label` is optional, non-unique display text. The artifact id is the stable identity for source media, generated media, documents, and final renders. |
 | `objects` | `object_hash TEXT`<br>`size_bytes INTEGER`<br>`created_at INTEGER`<br>`forgotten_at INTEGER?` | `object_hash PK`; `size_bytes CHECK >= 0`. Versioned metadata for bytes stored outside the database. Rows are append-only: `forgotten_at` marks a tombstone whose bytes were deleted by `deleteObject`/`gc`. |
 | `artifact_files` | `artifact_id TEXT`<br>`path TEXT`<br>`object_hash TEXT`<br>`mtime_ms INTEGER`<br>`created_at INTEGER` | `(artifact_id, path) PK`; `artifact_id FK → artifacts.artifact_id ON DELETE CASCADE`; `object_hash FK → objects.object_hash ON DELETE RESTRICT`. Maps a logical artifact path to immutable content. |
 | `book_metadata` | `key TEXT`<br>`value_json TEXT` | `key PK`. Singleton-book key/value metadata; no redundant `book_id`. |
@@ -120,7 +126,7 @@ There are 34 allowlisted semantic tables.
 | `entities` | `entity_id TEXT`<br>`type TEXT`<br>`name TEXT`<br>`description TEXT?`<br>`prompt TEXT?`<br>`data_json TEXT DEFAULT '{}'`<br>`created_at INTEGER` | `entity_id PK`; `type CHECK IN (prompt, character, scene)`. Normalized reusable creative concepts. |
 | `notebooks` | `notebook_id TEXT`<br>`name TEXT`<br>`created_at INTEGER` | `notebook_id PK`. Owns a generation/authoring graph. Schema v18 removed the monolithic `properties_json` cell. |
 | `notebook_fields` | `notebook_id TEXT`<br>`field TEXT`<br>`value_json TEXT` | `(notebook_id, field) PK`; notebook `FK → notebooks ON DELETE CASCADE`; `field` is restricted to the typed public notebook fields. Stores optional notebook-level workflow values independently. |
-| `cells` | `notebook_id TEXT`<br>`cell_id TEXT`<br>`type TEXT`<br>`slug TEXT`<br>`grid_row INTEGER`<br>`grid_column INTEGER`<br>`output_entity_id TEXT?`<br>`prompt TEXT?`<br>`provider TEXT?`<br>`model TEXT?`<br>`operation TEXT?`<br>`tool TEXT?`<br>`inputs_json TEXT DEFAULT '{}'`<br>`output_artifact_id TEXT?` | `(notebook_id, cell_id) PK`; `(notebook_id, slug) UQ`; notebook `FK → notebooks ON DELETE CASCADE`; entity and output artifacts use `ON DELETE RESTRICT`. Cell types and kind-specific slug prefixes are checked by the schema. |
+| `cells` | `notebook_id TEXT`<br>`cell_id TEXT`<br>`type TEXT`<br>`label TEXT?`<br>`grid_row INTEGER`<br>`grid_column INTEGER`<br>`output_entity_id TEXT?`<br>`prompt TEXT?`<br>`provider TEXT?`<br>`model TEXT?`<br>`operation TEXT?`<br>`tool TEXT?`<br>`inputs_json TEXT DEFAULT '{}'`<br>`output_artifact_id TEXT?` | `(notebook_id, cell_id) PK`; notebook `FK → notebooks ON DELETE CASCADE`; entity and output artifacts use `ON DELETE RESTRICT`. `label` is optional display text; the grid slot is the user-facing handle. |
 | `notebook_cell_executions` | `notebook_id TEXT`<br>`cell_id TEXT`<br>fingerprint/status/output/provider/run/timestamp/tool/error fields<br>`stale INTEGER`<br>`fixture_baseline INTEGER` | `(notebook_id, cell_id) PK` and composite cell FK with cascade. Gives each cell's execution and staleness state its own merge boundary. |
 | `notebook_generation_plans` | `notebook_id TEXT`<br>`plan_id TEXT`<br>`cell_id TEXT`<br>`status TEXT`<br>`plan_json TEXT`<br>output/error/timestamp fields | `(notebook_id, plan_id) PK`; composite cell FK with cascade. |
 | `notebook_run_plans` | `notebook_id TEXT`<br>`plan_id TEXT`<br>`status TEXT`<br>plan/cost/fingerprint/output fields<br>timestamps | `(notebook_id, plan_id) PK`. Stores approval and execution plans as independently mergeable rows. |
@@ -128,6 +134,7 @@ There are 34 allowlisted semantic tables.
 | `notebook_transcript_attachments` | `notebook_id TEXT`<br>`attachment_id TEXT`<br>`payload_json TEXT` | `(notebook_id, attachment_id) PK`; notebook FK with cascade. |
 | `edges` | `notebook_id TEXT`<br>`edge_id TEXT`<br>`source_cell_id TEXT`<br>`target_cell_id TEXT`<br>`target_input TEXT` | `(notebook_id, edge_id) PK`; notebook `FK → notebooks ON DELETE CASCADE`; composite source and target FKs reference cells in the same notebook and cascade on cell deletion. |
 | `runs` | `run_id TEXT`<br>`notebook_id TEXT`<br>`status TEXT`<br>`started_at INTEGER`<br>`completed_at INTEGER`<br>`cell_order_json TEXT`<br>`outputs_json TEXT`<br>`error TEXT?` | `run_id PK`; notebook `FK → notebooks ON DELETE CASCADE`; `status CHECK IN (completed, failed, aborted)`. Terminal, versioned notebook execution records. |
+| `generations` | `generation_id TEXT`<br>`notebook_id TEXT`<br>`cell_id TEXT`<br>`output_cell_id TEXT?`<br>`run_id TEXT?`<br>`status TEXT`<br>`tool TEXT`<br>`provider TEXT?`<br>`model TEXT?`<br>`prompt TEXT?`<br>`resolved_prompt TEXT?`<br>`provider_artifact_id TEXT?`<br>`output_artifact_id TEXT?`<br>`error TEXT?`<br>`created_at INTEGER`<br>`updated_at INTEGER` | `generation_id PK`; composite cell FK with cascade; `status CHECK IN (dispatched, awaiting_provider, completed, failed)`. One row per generation attempt; every transition is its own attributed semantic commit, so `dolt_history_generations` is the per-attempt timeline. |
 
 ### Sequence timeline and media editing state
 
@@ -229,17 +236,10 @@ are the constraint-verification primitives.
   `SCHEMA_INCOMPATIBLE` before any merge is attempted
   (`assertSameSchemaVersion`). This was previously only implied by the
   engine's open-time version gate.
-- **`artifacts.slug` (globally unique) → user-facing conflict.** Two forks
-  minting the same slug for different artifacts is detected before the
-  merge by a three-way (merge-base/ours/theirs) simulation over the
-  artifacts slug projection (`findSlugConflicts`, using `dolt_merge_base`
-  and `dolt_at_artifacts`), and refused with `MERGE_CONFLICT` listing the
-  slug and both artifact ids. doltlite's own verification is the backstop:
-  refusals it reports are re-diagnosed and surface as `MERGE_CONFLICT`
-  when attributable to slugs. Row-level same-row edits surface as
-  `MERGE_CONFLICT` as well. At create time, slug dedup runs inside the
-  serialized write chain (not as a read-then-write outside it), so
-  concurrent creates cannot race onto the same slug.
+- **Artifact identity is `artifact_id` (UUIDv7) → no name-conflict
+  class.** Forks mint collision-free ids, and `artifacts.label` is
+  non-unique display text that merges as an ordinary column. Row-level
+  same-row edits surface as `MERGE_CONFLICT`.
 - **RESTRICT foreign keys → verification-surfaced typed violation.** A
   fork that deletes a row another fork newly references is caught by
   doltlite's merge-time working-set verification, which refuses and rolls
@@ -271,19 +271,12 @@ are the constraint-verification primitives.
   both lineages) and two set values keep the earlier timestamp
   (`resolveObjectsRow` in `src/fork.ts`).
 
-ve-wsu: doltlite currently corrupts secondary UNIQUE indexes on
-`dolt_checkout` once a working set has three or more tables, corrupts full
-engine catalogs on checkout and `dolt_clone` (the cloned file's schema does
-not even parse: `invalid rootpage` on a secondary autoindex), misfires its
-"uncommitted changes" merge guard on the full 28-table catalog (every
-table reports a phantom `modified` status with zero row diffs), and — when
-the guard is bypassed by committing the phantom dirt — dies in schema
-loading on true merges (`schema conflict on table 'sqlite_autoindex_*'`).
-The dedicated merge-back flow therefore runs this policy around a
-projection-level three-way merge instead of `dolt_merge`; see "Forks and
-merge-back integration" below. `mergeWithPolicy` remains the drop-in merge
-mechanism once the upstream bugs are fixed, exercised against the real
-semantic DDL in `tests/merge-policy.test.ts`.
+DoltLite 0.50.6 fixes incremental staging and full-catalog URL cloning.
+Native merge still refuses a catalog containing ignored runtime tables,
+even with an empty semantic diff and, without a secondary runtime index,
+an empty `dolt_status`. `ve-wsu` retains this gate; see the reproductions in
+[docs/doltlite-staging.md](doltlite-staging.md). The dedicated merge-back flow
+keeps its projection merge and application conflict policies.
 
 ### Forks and merge-back integration
 
@@ -292,14 +285,13 @@ A fork of a public book is, from the engine's point of view:
 1. **A platform fork.** Creating the hosted copy of a catalog and giving
    it a URL is the hosting layer's job; it is out of engine scope.
 2. **A clone of the catalog into a local engine root** (`bootstrapFork` in
-   [`src/fork.ts`](../src/fork.ts)). Because `dolt_clone` corrupts full
-   catalogs (ve-wsu), bootstrap takes a byte snapshot of a healthy upstream
-   `videobook.db` (captured while the upstream engine is closed) and opens
-   it as a normal engine — no `initialBookSlug`, the singleton book row
-   comes along with the snapshot. A URL bootstrap path attempts
-   `dolt_clone` and health-validates the result, surfacing a typed
-   `FEATURE_UNAVAILABLE` while the upstream bug stands; it starts working
-   unchanged once doltlite is fixed.
+   [`src/fork.ts`](../src/fork.ts)). A URL bootstrap uses `dolt_clone` and
+   validates the resulting schema before opening the engine. It also accepts
+   a byte snapshot of a healthy upstream `videobook.db` captured while the
+   upstream engine is closed. Both preserve the singleton book row without
+   requiring `initialBookName`. The upstream staging fix does not repair
+   commits corrupted by older dependencies; those catalogs may still need
+   snapshot bootstrap.
 3. **A public-read object store keyed by SHA-256.** `ContentStore` stays
    the abstraction; the existing `ensureLocal` lazy fetch in
    [`src/cas.ts`](../src/cas.ts) downloads any object the fork lacks on
@@ -323,10 +315,10 @@ Integration is a dedicated flow, `mergeBack` in
    resolve heads and the merge base (`dolt_merge_base`; commit hashes via
    `dolt_log`/`dolt_branches` — `doltHashOf` returns content hashes, not
    commit hashes; the fetched remote-tracking ref gets a local branch
-   pointer, a ref-only write that is safe under ve-wsu).
-3. Run the merge policy: same-schema precondition, pre-merge slug-conflict
-   detection (`MERGE_CONFLICT`), then a projection-level three-way row
-   merge over `dolt_at_<table>` snapshots of base/ours/theirs (row
+   pointer without checking out the fetched branch).
+3. Run the merge policy: same-schema precondition, then a
+   projection-level three-way row merge over `dolt_at_<table>`
+   snapshots of base/ours/theirs (row
    semantics mirror Dolt: one-sided changes win, identical changes
    resolve, incompatible changes abort with `MERGE_CONFLICT`), deterministic
    singleton-flag reconcile, and post-merge constraint verification
@@ -338,14 +330,12 @@ Integration is a dedicated flow, `mergeBack` in
    `engine.storage.backup`.
 5. Land one forward integration commit on `main` and `dolt_push` it.
 
-ve-wsu makes a true two-parent `dolt_merge` commit impossible on full
-catalogs today, so the integration commit is single-parent and records the
-integrated fork head in a `merged-revision` commit-message trailer (plus
-`base-revision`, per the structured-message convention). Re-running the
-flow is a no-op when the fork's net changes are already on `main`. The
-projection merge in `mergeRefs` is the single swap point: when doltlite is
-fixed, `mergeWithPolicy` + `dolt_merge` replace it and the commit becomes
-a true merge commit.
+The integration commit remains single-parent and records the integrated fork
+head in a `merged-revision` commit-message trailer (plus `base-revision`, per
+the structured-message convention). Re-running the flow is a no-op when the
+fork's net changes are already on `main`. A future native replacement must
+pass the ignored-runtime merge gate in `ve-wsu` and retain singleton
+reconciliation, forget-wins object handling, and object publication ordering.
 
 When a plain `engine.storage.backup()` push is rejected because upstream
 moved, the backup surfaces `DIVERGED` with guidance into this flow —
@@ -370,6 +360,7 @@ schema additionally defines every index below.
 | `runs_notebook_completed` | `runs(notebook_id, completed_at, run_id)` |
 | `prompt_entries_lookup` | `prompt_entries(surface, created_at, prompt_id)` |
 | `messages_created` | `messages(created_at, message_id)` |
+| `generations_cell` | `generations(notebook_id, cell_id, created_at)` |
 
 ## Local-only runtime schema
 
@@ -495,7 +486,22 @@ the engine deliberately does not implement it.
 - doltlite exposes `dolt_gc()` as a SQL function (verified: it returns a
   `"N chunks removed, M chunks kept"` summary). `gc({ doltGc: true })` runs
   it after collecting to physically reclaim chunks left behind by dropped
-  table data in the versioned catalog.
+  table data in the versioned catalog. The store also GC's automatically
+  at open when `videobook.db` exceeds 64 MiB without a verified compaction
+  record (configurable via
+  `EngineConfig.catalogGc`) and at close after any runtime or semantic
+  write, returning a `CatalogGcReport` (`engine.lastCatalogGc` /
+  `engine.gcCatalog()`) with the summary and byte delta. GC never mints a
+  commit. Periodic GC-after-N-writes is not implemented: cached prepared
+  statements would have to be dropped, and `dolt_gc` cannot run inside
+  `serial()` or an open transaction.
+- After successful GC, a clean close atomically writes `videobook.db.gc.json`
+  with the catalog's device, inode, size, and nanosecond modification/change
+  times. An unchanged catalog can then skip repeated open-time compaction.
+  Transactions invalidate this disposable record before writing. Missing,
+  malformed, mismatched records and nonempty WAL/journal files preserve the
+  size-triggered GC fallback. Runtime schema metadata is only rewritten when
+  its version changes, avoiding a write on every read-only open.
 - Run `deleteObject` and `gc` only while no imports are in flight; CAS puts
   happen outside the serialized write chain, so a concurrent import could
   race the sweep.
@@ -542,7 +548,7 @@ interface ArtifactManifestFile {
 
 interface ArtifactManifest {
   artifactId: string;
-  slug: string;
+  label?: string;
   path: string;               // disposable workspace path
   fileCount: number;
   files: ArtifactManifestFile[];
@@ -701,7 +707,7 @@ must reproduce a render.
   selections, UI panels, decode caches, thumbnails, temporary renders, active
   jobs, locks, and presence in ignored runtime tables.
 - Pin renders and derived analysis to source object hashes and a Dolt revision,
-  not mutable artifact slugs or workspace paths.
+  not mutable artifact labels or workspace paths.
 - Treat color space, transfer function, matrix, range, alpha mode, rotation,
   sample rate, channel layout, and variable-frame-rate timing as first-class
   media facts.

package/docs/media-limits.md

@@ -0,0 +1,57 @@
+# Local media decoding limits
+
+Temporal CLIP/CLAP and compatibility similarity use shared bounded decoding
+helpers. FFmpeg and FFprobe run with argument arrays and no shell. Their
+default execution deadline is 120 seconds. A caller can provide a positive,
+finite `timeoutMs` and an `AbortSignal` through `MediaOperationOptions`.
+Cancellation, deadline expiry or excessive output kills the process and waits
+for its close before rejecting, allowing callers to clean temporary workspaces.
+
+Standard error is capped at 64 KiB. Probe output is capped at 64 KiB; PCM
+output is capped at the requested sample count times four bytes. Temporal
+CLAP decodes at most ten seconds of 48 kHz mono audio. Other process output
+has an 8 MiB default cap. Decoder failures report typed errors and exit
+information without copying raw stderr, input paths or cancellation reasons
+into the error message. `TIMEOUT`, `CANCELLED`, `RESOURCE_EXHAUSTED`,
+`FEATURE_UNAVAILABLE` and `INVALID_INPUT` distinguish failure categories.
+
+FFmpeg input protocols are restricted to `file,pipe`. Compatibility video
+extraction uses a disposable directory, at most 120 PNG frames and a fixed
+224 by 224 output size, with cleanup in `finally`. These controls do not
+constitute a filesystem sandbox for a decoder.
+
+Image inputs must be regular local files no larger than 64 MiB and 40 million
+pixels. Sharp reads only the first image, applies a processing timeout and
+converts to RGB using the existing orientation and colour normalization.
+Accepted images keep the same preprocessing; oversized images fail before
+an unbounded RGB allocation. The pixel cap also limits expansion of compact
+inputs such as large SVG canvases. No global Sharp settings are changed.
+
+Compatibility preparation/index/query methods forward media options to their
+providers. Indexing checks cancellation before publishing runtime coverage.
+Temporal providers accept media options directly:
+
+```ts
+await clip.embedImage(path, { signal });
+await clap.embedAudio(path, 0, 10, { signal, timeoutMs: 30_000 });
+await engine.similarity.index(artifactId, { signal });
+```
+
+`tests/media-limits.test.ts` uses actual child processes for stalled/noisy
+output, literal arguments, abort-before-spawn, running cancellation and safe
+errors. Real Sharp fixtures cover malformed, oversized and high-expansion
+images. A scoped engine fixture verifies failed decoding leaves source data
+and prior coverage intact and allows retry. The opt-in cached CLIP/CLAP and
+compatibility E2E tests verify real JPEG/MP4/WAV decoding, CLAP timeout and
+cancellation, malformed audio and successful retry.
+
+Built-in model loading, inference and Sharp decoding now run in the isolated
+process pool described in `docs/model-isolation.md`. Its outer deadline and
+process-group cancellation cover native work that cannot observe an AbortSignal
+inside the call. Sharp's own processing deadline is rounded up to seconds.
+Consumer `09647d69` forwards job cancellation through its provider overrides
+and indexing/reference calls. Actual queue tests cancel stalled model requests,
+preserve source bytes and ready status, fail malformed-image indexing with a
+typed error, and successfully index a corrected retry. Audio-only indexing uses
+CLAP without passing its source to the image decoder. A cancelled completed
+batch is rejected before coverage publication, preserving the last cursor.

package/docs/model-download-policy.md

@@ -0,0 +1,15 @@
+# Local model download policy
+
+Temporal CLIP/CLAP and compatibility image, audio, and text similarity loaders use local files unless `allowModelDownload` is explicitly `true`. A missing or unusable model returns `OFFLINE` when downloads are disabled. An application should offer an explicit preparation step, then use the populated cache for ordinary search and indexing.
+
+The pinned Transformers.js 4.2.0 dependency drops `cache_dir`, `local_files_only`, and `revision` during metadata discovery in `pipeline`, `loadTokenizer`, and the model-registry file-list helpers. Without correction, even cached or local-only preparation can request metadata from `main`.
+
+`scripts/patch-transformers.mjs` corrects option forwarding in the dependency's Node ESM entry before tests and bundling. It checks the dependency version and exact source fragments, is idempotent, and fails when those fragments change. The packaged runtime includes this correction; applications do not need Transformers.js installed separately. The third-party version manifest records `model-discovery-options-v1`. Review and remove the patch when a verified upstream version preserves these options throughout discovery.
+
+`tests/model-download-policy.test.ts` uses a real HTTP server to count requests. It checks zero requests with absent or false permission across temporal and compatibility providers, and pinned requests with explicit permission. The real-model transfer test serves the existing pinned snapshots from a local fixture cache into an empty destination, then prepares new providers and repeats inference with downloads omitted. It requires an explicit test opt-in and never accesses the public Hub:
+
+```bash
+VIDEOBOOK_RUN_MODEL_POLICY_E2E=1 npm test -- tests/model-download-policy.e2e.test.ts
+```
+
+The source fixture cache defaults to `~/.cache/videobook/models`; override it with `VIDEOBOOK_MODEL_FIXTURE_CACHE`. It must already contain the pinned CLIP and CLAP snapshots used by the engine. This Transformers.js cache layout differs from the Hugging Face Hub CLI cache layout.

package/docs/model-integrity.md

@@ -0,0 +1,72 @@
+# Model file integrity
+
+Built-in model workers verify model files before the loader receives either
+bytes or an ONNX file path. This covers configuration, tokenizers, processors,
+model graphs and external ONNX weight shards. A verification error returns
+`MODEL_UNAVAILABLE`; no corrupted download is published to the model cache.
+Integrity failures remain fatal even if an upstream optional-file loader catches
+the initial exception. They are not reported as missing-model readiness.
+
+`src/model-checksums.json` records the complete file inventories of the three
+built-in pinned CLIP, CLAP and MiniLM snapshots. The hashes and sizes come from
+the Hugging Face repository tree at each exact revision. Git files use Git blob
+SHA-1, including the `blob <byte-count>\0` prefix; LFS files use content SHA-256.
+The distinction follows the [Hub file metadata contract](https://huggingface.co/docs/huggingface_hub/en/package_reference/file_download).
+Existing built-in caches can therefore be verified offline without adding a
+network dependency or changing the Transformers cache directory layout.
+
+The checksums are for these revisions:
+
+- [CLIP d15189d7](https://huggingface.co/Xenova/clip-vit-base-patch32/tree/d15189d7028b43f1d3e65039190477f6af591c2a)
+- [CLAP c28f2883](https://huggingface.co/Xenova/clap-htsat-unfused/tree/c28f2883575e590e04d3146ff0713c2448d691ba)
+- [MiniLM aff7a1dc](https://huggingface.co/onnx-community/all-MiniLM-L6-v2-ONNX/tree/aff7a1dc4e8a1ea593e6ea21e95c22ef0a25966f)
+
+Maintainers can regenerate the inventories with
+`node scripts/update-model-checksums.mjs`, using the installed `hf` CLI. Review
+the generated diff and keep model revision constants and inventories aligned.
+Builds and ordinary model use never run this networked maintenance command.
+
+For other remote models, the resolver checks a supported strong ETag or
+`X-Linked-ETag` on the original model response. Redirects retain that digest;
+CDN ETags and Xet chunk identifiers are not substituted for the original file
+hash. Authorization tokens are sent only to `https://huggingface.co`, and
+redirects cannot downgrade HTTPS. If upstream supplies no supported checksum,
+the cache receipt records a transport SHA-256 for subsequent corruption
+detection, explicitly distinct from upstream verification.
+
+Custom remote-model cache files without integrity metadata require an explicit
+online preparation before offline use. Explicit local model directories remain
+application-trusted inputs with no implied upstream authenticity. Local cache
+receipts detect corruption; they do not defend against someone deliberately
+rewriting both a custom model and its receipt with local filesystem access.
+
+Each worker reads a cached file into its own verified snapshot before use. The
+loader never receives the original mutable cache path. Relative names and
+directory layout are retained so ONNX external data stays adjacent to its graph.
+Loaded models and verified snapshots are reused within that worker. A new worker
+verifies cached bytes again. Changing a cache file cannot change an already
+verified worker snapshot.
+
+Downloads stream into the owned worker workspace, are verified, then are copied
+to an owned staging directory under the configured cache and atomically renamed.
+Receipts are published separately; interrupted publication can require another
+preparation for a custom model but cannot authorize mismatching bytes. Parent
+cancellation, eviction and exit remove owned worker and cache-staging files.
+Files are capped at 2 GiB and in-memory metadata returns at 16 MiB. The existing
+worker deadlines, process limits and cancellation cover hashing and transfer.
+
+An invalid existing cache entry is left intact for diagnosis. Remove or restore
+the affected entry, then explicitly prepare the model to retry. Preparing does
+not silently delete a corrupt file. The Hugging Face CLI's `hf cache verify`
+supports `--local-dir` pointing at one Transformers model/revision directory
+when an administrator needs a file-level audit.
+
+Tests cover Git and LFS digests, redirects, unsupported metadata, corrupt
+downloads and receipts, offline corruption, concurrent loads, snapshot isolation,
+path/size limits, and killed cache-staging cleanup. Real-model tests transfer and
+reuse CLIP/CLAP, reject modified JSON/tokenizer/ONNX bytes, and exercise MiniLM
+external weights, offline reopen and a corrupt weight shard. The installed
+package smoke always checks corrupted pinned configuration rejection and can
+also run real cached model inference.
+
+Custom remote compatibility models also require an explicit immutable commit. See [model revision selection](model-revisions.md) for coherent file loading, embedding-space isolation and migration from moving aliases. Explicit local-directory models remain trusted local inputs.

package/docs/model-isolation.md

@@ -0,0 +1,72 @@
+# Local model process isolation
+
+Built-in temporal CLIP/CLAP and compatibility image/video/audio/text providers
+load models and perform inference in child processes. Workers receive only
+the selected text or absolute media path and a projected model configuration.
+They do not receive an Engine, a catalog handle, book metadata, job payloads
+or general provider credentials, and never open a book database.
+
+Each calling Node process shares a pool of at most two model workers, keyed
+by model/cache/download configuration. Calls to one worker are serialized;
+the pool accepts at most 64 waiting calls. Matching workers reuse loaded models.
+Idle workers are evicted when another configuration needs capacity or after
+30 seconds. They do not keep the host process alive. Host exit kills cached
+workers and removes their owned scratch directories.
+
+The worker environment includes basic executable/home/temp/locale settings.
+Hugging Face token variables are forwarded only when downloads are explicitly
+enabled. Provider keys, arbitrary environment variables and `NODE_OPTIONS`
+are excluded. The existing Transformers remote-host/local-model settings are
+projected into the worker so pinned offline discovery and configured mirrors
+retain their behavior. Arbitrary JavaScript fetch hooks are not serialized.
+
+Preparation defaults to a 15-minute deadline, covering explicit first-time
+downloads. Inference defaults to two minutes, including waiting for a worker.
+`MediaOperationOptions.timeoutMs` overrides the deadline; `signal` cancels a
+queued call without affecting active work, or kills the active worker and its
+decoder process group. A request resolves or rejects after required worker
+cleanup. The next call after a failure starts a fresh process. Applications
+must forward their job signal through provider overrides and calls.
+
+Workers have a 512 MiB V8 heap limit. Model weights and native allocations are
+outside that heap limit; it is not a total-RSS operating-system sandbox.
+Input text is capped at 1 MiB, media paths at 4 KiB, IPC messages at 2 MiB and
+diagnostic output at 64 KiB. Returned vector dimensions, finite values, text
+offsets and video frame counts are validated before use. Media decoding also
+uses the limits in `docs/media-limits.md`.
+
+A fatal worker allocation failure or abrupt termination returns a typed
+`RESOURCE_EXHAUSTED` or `MODEL_UNAVAILABLE` error to the host. Ordinary allocator
+errors are classified as resource exhaustion even when a model loader wrapped
+them in an offline-readiness error. Unrecognized exceptions do not forward
+model inputs or raw native diagnostics. No semantic transaction spans an
+inference call, and cancellation is checked before index coverage publication.
+
+`tests/model-worker-pool.test.ts` uses actual child processes to verify reuse,
+capacity, queue cancellation, blocked execution, decoder descendants, invalid
+responses, environment filtering, idle cleanup and host-exit cleanup. A worker
+with a deliberately small heap is exhausted while a real engine remains open;
+the test then writes, reopens the book and successfully starts another worker.
+An additional durable QueueRunner test records the allocation failure as a
+failed job with `RESOURCE_EXHAUSTED`, then completes another job successfully.
+QueueRunner preserves EngineFault codes instead of replacing them with the
+exception class name.
+This is a real V8 allocation failure, not a simulation of physical disk failure
+or whole-machine memory pressure. `tests/local-models.e2e.test.ts`, compatibility
+media E2E and the model-transfer fixture exercise actual models through the
+same transport. The package smoke also starts the installed JavaScript worker,
+checks missing-cache offline failure, and can run cached CLIP/CLAP inference.
+
+The worker entry is compiled into the package and declared as an executable
+entry in the dead-code configuration. Source tests use the installed `tsx`
+loader; installed packages use JavaScript and need no `tsx` dependency.
+
+Consumer `09647d69` vendors engine `da41904` and forwards cancellation through
+all provider overrides and index/reference jobs. Local HTTP fixtures hold model
+requests open until actual queue cancellation kills the worker and closes its
+connections. Preparation failures and cancellation leave source artifacts ready
+and byte-identical. Reference-preparation jobs are detached from source artifact
+status, as indexing jobs already were. Missing-model offline readiness remains
+deferred; execution failures fail the auxiliary job with their engine code.
+Cached-model queue tests cover malformed-image failure and corrected retry, plus
+audio-only indexing without invoking the image decoder.