videobook-engine 4.2.0 → 5.3.2

package/docs/model-revisions.md

@@ -0,0 +1,54 @@
+# Immutable model revision selection
+
+Built-in CLIP, CLAP and MiniLM models continue to use their existing pinned commits. No new model setting is needed for their default configuration. The compatibility image/video vector cache requires a one-time reindex, as described below. Temporal, audio and text default embedding identities are unchanged.
+
+For a custom remote model used by the compatibility similarity API, supply `modelRevision` alongside `modelId`. It must be a full 40-character hexadecimal commit SHA from that model repository. Branch names, tags, abbreviated hashes and omitted custom revisions fail with `INVALID_INPUT` before any model request. Uppercase hashes are normalized to lowercase.
+
+```typescript
+const engine = createEngine({
+  rootDir,
+  similarity: {
+    modelCacheDir,
+    modelId: "your-org/compatible-clip",
+    modelRevision: clipCommitSha,
+    allowModelDownload: true,
+    audio: {
+      modelId: "your-org/compatible-clap",
+      modelRevision: clapCommitSha,
+    },
+    text: {
+      modelId: "your-org/compatible-minilm",
+      modelRevision: textCommitSha,
+    },
+  },
+});
+const prepared = await engine.similarity.prepare();
+```
+
+Each `*CommitSha` variable above is the actual full commit ID selected by the application. The engine does not resolve a moving alias implicitly. Supplying another commit for a built-in repository is also supported and creates a distinct custom embedding identity. Audio and text select their own repositories and revisions; they do not inherit the visual model selection. The existing cache-directory and download-permission inheritance rules are unchanged.
+
+Custom models must remain compatible with the corresponding pipeline: 512-dimensional CLIP image/video embeddings, 512-dimensional CLAP audio embeddings or 384-dimensional MiniLM text embeddings, with the supported quantization and preprocessing. Selecting a commit does not make an incompatible model architecture compatible.
+
+## Coherent loading and index identity
+
+The selected repository and commit are copied into the model worker configuration and worker-session key. Every model-file request must agree with them, including tokenizer/config files and external ONNX weights. A request for another repository, another commit or an omitted/moving revision fails with `MODEL_UNAVAILABLE`. Remote selections do not fall back to unversioned files under Transformers' `localModelPath`; use an explicit directory model ID for trusted local files.
+
+Custom embedding-space identities include the provider kind, exact repository, full revision and preprocessing version through a SHA-256 digest. Repository names that previously collided after punctuation normalization now remain distinct. A different revision cannot reuse vectors from an existing space, even while the old model worker is warm. Changing a caller's configuration object after Engine construction does not alter the selected model.
+
+Existing custom-model vector identities intentionally change. Reindex custom-model artifacts after supplying the commit. The API shape remains source-compatible on version 5.3.1, but old custom remote configurations without a commit now fail closed rather than loading `main`.
+
+Older compatibility image/video vectors are ambiguous: custom image models previously used the same embedding-space string as the built-in CLIP model, without recording which model produced a row. The compatibility visual space now uses `compat-visual-v2`, including for the default model. This requires a one-time reindex of compatibility image/video artifacts and prevents old custom vectors from silently being treated as built-in vectors. Until reindexed, status returns `not_indexed` and similarity queries return `NOT_READY`. Source media and the old derived rows remain intact. The media library's temporal index and the default compatibility audio/text spaces retain their identities; model files do not need to be downloaded again.
+
+Model downloads still require explicit permission. After online preparation, the selected revision's verified cached files and integrity receipts support offline preparation, indexing and queries. Changing to a revision that is not cached returns `OFFLINE`; it does not substitute another revision. [Model integrity verification](model-integrity.md) describes digest verification and its trust boundary for custom repositories.
+
+## Local directories
+
+Absolute paths and explicit relative paths beginning with `.` remain supported as trusted local model directories, without `modelRevision` or network access. Relative paths are resolved when the provider is configured. Equivalent resolved paths share an identity; different directory paths have distinct identities. `modelRevision` applies only to remote repositories and is rejected for directory selections.
+
+Keep each local model version in a separate immutable directory. Local directories do not carry a verified upstream commit, and their contents are not authenticated as a remote snapshot. Replacing files in place requires application-owned reindexing; the engine does not claim that a path alone makes local bytes immutable.
+
+Injected embedding providers manage their own model identity and revisions. These options apply to built-in loader implementations, not to arbitrary injected code. Injected providers remain subject to the [network declaration and application consent contract](search-provider-consent.md).
+
+## Verification
+
+`tests/model-revision.test.ts` uses actual Engine APIs and an HTTP receiver to cover rejected revisions with zero requests, all three modalities' pinned requests, separate identities, configuration snapshots, local-directory selection, and cross-repository/revision file rejection. `tests/model-revision.e2e.test.ts`, enabled by `VIDEOBOOK_RUN_CUSTOM_MODEL_E2E=1`, serves real cached CLIP/CLAP/MiniLM files under custom repository IDs and verifies coherent revision requests, indexing, offline reopen, revision isolation, retained source bytes and explicit local-directory inference. It requires the three fixture models to have been prepared separately; it does not download them from an external service.

package/docs/mvp-prd.md

@@ -1,7 +1,7 @@
 # videobook-engine MVP Requirements
 
 > Historical design baseline: this document proposed the 3.x/schema-v5
-> transition. The implemented package is 4.0.0 with catalog schema v18; current
+> transition. The implemented package is 4.2.0 with catalog schema v19; current
 > API and layout documentation take precedence.
 
 ## Temporal media retrieval and a reversible edit engine
@@ -365,7 +365,7 @@ runtime structures capable of many segments and modalities per artifact:
 | Segment text | Transcript/OCR/description/metadata text, language, half-open UTF-8 byte offsets or timed source ranges, confidence, provenance |
 | Segment embeddings | Segment, modality, embedding space, dimensions, quantized vector, source hash, updated time; multiple spaces permitted |
 | Exact/near duplicates | Cryptographic hash and optional perceptual/frame fingerprints with extractor version |
-| Lexical index | FTS-capable index over transcript, OCR, descriptions, labels, slugs, and selected metadata |
+| Lexical index | FTS-capable index over transcript, OCR, descriptions, labels, and selected metadata |
 | ANN indexes | Cached per embedding space and compatible query modality, rebuilt from runtime rows |
 
 All runtime tables remain excluded from Dolt staging. A stale manifest or source

package/docs/release-evidence.md

@@ -0,0 +1,300 @@
+# MVP release evidence
+
+Audit date: September 6, 2026 (Asia/Bangkok). Functional baseline: engine
+`c4f1d89` (5.3.2, DoltLite 0.50.6), consumer `92f1f2c6`.
+Performance artifacts retain their original revision and hardware
+qualifications.
+This is an assessment of the requirements in
+`docs/mvp-prd.md`, not a release approval. E4, E5 and the MVP remain incomplete.
+Beads contains the work assignments and current status; this report records
+the evidence and its limits at these revisions.
+
+## Current patch candidate
+
+Engine `c4f1d89` adopts DoltLite 0.50.6. Both default and all-table staging
+probes now pass on the real 12-table fixture, and URL bootstrap preserves the
+complete catalog. Catalogs seeded with the previous engine's 0.11.37 and the
+consumer's 0.11.51 reopen with identical committed head, history, table counts,
+semantic projections, contents, and runtime settings; restore and new writes
+remain durable after another reopen.
+
+Native merge still rejects clean catalogs containing ignored runtime tables.
+`scripts/dolt-ignored-merge-probe.cjs` reproduces this even without secondary
+indexes; its semantic-only control succeeds. Production retains the projection
+merge and its existing policies. `ve-wsu` remains blocked on the native merge
+limitation; the staging and URL-bootstrap fixes do not close it.
+
+The engine passes 352 default tests, typecheck, dead-code checks, build, and
+clean packed README/reopen/media checks, including cached CLIP/CLAP image,
+audio, and text inference. The package has one Sharp resolution and no
+production audit findings. Consumer `92f1f2c6` passes 3,218 tests across 375
+files, lint, all four test typechecks, dead-code checks, and client/server builds.
+Its explicit cached-model, readiness, bounded-batch/resume, and reference-range
+run passes all 24 cases on `92f1f2c6`. Its isolated exact committed-source
+clean install passes native Sharp,
+MCP create/list, client delivery, and graceful shutdown without a sibling engine
+checkout. The consumer corpus validator now rejects unverified rights metadata,
+non-finite durations/counts, and fractional counts; its runner is typechecked
+against the current book identity API. These synthetic manifest checks do not
+supply a quality corpus.
+
+The application CI run exposed three additional issues, corrected in
+`92f1f2c6`. Watermark text is now centered by its rendered glyph bounds rather
+than platform-specific SVG baseline metrics; changed badge variants use new
+cache layout identifiers. Queued watermark burns declare processing work, and
+upload discovery respects active queued jobs even before they claim a lock.
+This prevents a second processing job from racing the watermark operation and
+removing its frame directory during ingestion. The GC fixture stops its
+background worker before creating 2,000 leased churn rows, avoiding a large
+expired-job abort sweep during teardown. Its size assertions are unchanged.
+The original rendering and queued-job failures reproduce on Linux; all 41
+focused cases pass after the fixes on both macOS and Debian 13/x64. The full
+application CI command passes locally in both stages (3,214 plus four tests),
+with all typechecks, lint, dead-code checks and both builds passing.
+
+The catalog-GC churn case allows 180 seconds for its 2,000 synchronous job
+write/dequeue/heartbeat iterations on shared CI runners. Its 200 ms reopen and
+16 MiB catalog-size assertions are unchanged. The focused seven-case suite
+passes locally; the timeout change addresses the 84-second fixture that
+exceeded the previous 60-second watchdog on Node 22.
+
+The candidate is a patch within the existing 5.x series. No 2.0.0 release or
+major bump is planned. Registry publication and frozen-corpus quality remain
+outstanding. The M2 Pro reference-device run is deferred because the user
+confirmed that the hardware is unavailable.
+
+## Performance and quality
+
+The available machine reports Apple M1 Pro, 10 logical CPUs and 16 GiB RAM.
+The PRD specifies M2 Pro, 16 GB RAM and local SSD. Existing measurements retain
+their actual hardware qualification; none is an M2 Pro acceptance run. The
+reference-device measurements are deferred in `ve-ovz.22` following the user's
+September 6 confirmation that no M2 Pro is available. The passing M1 Pro results
+remain the available benchmark evidence. This exact-device check does not
+block publication of the already validated 5.3.2 patch.
+
+| Requirement | Evidence inspected | Assessment and follow-up |
+| --- | --- | --- |
+| VE-NFR-001: 1,000-artifact open and semantic summary <2 s | Clean engine `c4f1d89`: 171 ms same-process after a fresh 1,000-artifact/100,000-moment build and 1,714 ms in a fresh process reopening the retained fixture. Raw reports: `temporal-100k-doltlite-0506.json` and `temporal-100k-doltlite-0506-reopen.json`, both on M1 Pro. | Passes on measured hardware for the current source. The M2 Pro reference-device run remains outstanding and is tracked explicitly; M1 Pro results are not equated with it. |
+| VE-NFR-002: metadata and imported normalized transcript searchable <5 s after semantic commit | Consumer `tests/v2/text-readiness.test.ts` times the real job queue on a real probed video: analysis metadata write to first lexical hit 688 ms (OCR text 689 ms), transcript import to first quoted speech hit 314 ms, both under the 5 s gate with model downloads disabled and no cached model. | Passes end to end through the application's actual enqueue, poll, index and lexical query path on M1 Pro. |
+| VE-NFR-003: searchable coverage at least every 60 source seconds, resume last committed batch | Synthetic benchmark: 4,000 batches of at most 30 source seconds, every cursor checked, first coverage 45 ms. The consumer's fixed four-unit deep batches could exceed 60 s on videos longer than about seven minutes because frame sampling caps at 30 frames; `commitDeepBatches` now bounds each CLIP/CLAP batch by covered source seconds. Consumer `tests/v2/semantic-index-cadence.test.ts` verifies the bound and durable-cursor resume through the engine, and the explicit real-model case runs the actual CLIP pipeline on a real ten-minute video: 15 two-frame batches of at most 41.4 s, interruption after two batches, resume from cursor 4 without re-embedding committed frames, and retrieval of the source. | Passes for the actual model pipeline. A single sampled frame whose own span exceeds 60 s (videos over about 29 minutes) is committed alone; frame density itself is a quality question for the frozen corpus, not a cadence failure. |
+| VE-NFR-004: 100k moments, warm p50 <500 ms, p95 <1.5 s including hybrid | Clean `c4f1d89` fresh build: 50 warm reads per mode, p50/p95 image 59/66 ms, video 113/165 ms, hybrid 332/411 ms; fresh-process reopen: image 60/73 ms, video 117/179 ms, hybrid 310/388 ms. | Passes the full synthetic workload on the current source, M1 Pro. Reference-device run outstanding and tracked explicitly. |
+| VE-NFR-005: 100-operation preview on 1,000 clips, p95 <250 ms, no mutation | Clean `c4f1d89`, `npm run benchmark:edits` (`docs/edit-performance.md`): 50 independent previews of fresh 100-operation batches against 1,000 clips, p50 105 ms, p95 114 ms, max 118 ms; head revision, every table row count and the canonical sequence projection unchanged after each preview. Recorded in `benchmarks/results/edit-100x1000-doltlite-0506.json`. | Passes with strict gates on M1 Pro. `tests/edit-transactions.test.ts` keeps its single-sample 500 ms tripwire for shared CI runners. |
+| VE-NFR-006: same commit batch, p95 <1 s | Same run: 50 independent commits, p50 299 ms, p95 331 ms, max 348 ms; every commit advanced the head and applied all 100 transforms; reopened catalog exposes the last revision. | Passes on M1 Pro; no derived jobs run inside `edits.commit`. |
+| VE-NFR-007: 100k query/index structures <4 GB RSS beyond loaded model | Clean `c4f1d89` fresh build and query process peak 3.03 GiB including fixture construction; fresh-process reopen and 150 queries 2.08 GiB; no model loaded. | Passes on the current source, M1 Pro. Reference-device run outstanding. |
+| VE-NFR-008: forced termination at every SQL/outbox/Dolt boundary | Baseline edit tests only threw exceptions and closed normally. Subsequent `tests/semantic-crash.test.ts` covers real SIGKILL at each semantic/outbox/table-staging/Dolt boundary for a multi-table edit and provenance operation, including interrupted recovery and an intervening write. | Kill matrix exposed and corrected duplicate provenance replay. See `docs/semantic-durability.md` for scope and invariants; tracked in `ve-ovz.10`. |
+| VE-NFR-009: stable search ordering, identical canonical previews/hashes | Current-source temporal runs repeat every first query per mode against the unchanged generation and require identical hits; the edit distribution run previews each of 50 large batches twice and requires identical canonical operations, ranges, write sets, diffs and all hashes. | Passes for the exact 100-operation/1,000-clip workload and the 100k search workload. |
+| VE-NFR-010: every application frozen-corpus quality threshold | E4 evaluator and small real-model fixtures exist | Full rights-cleared frozen corpus and judged ranges absent. `ve-s84` remains incomplete; synthetic scale data cannot replace it. |
+| VE-NFR-011: with required models cached, search/index/edit/history initiates no network | Per-Engine local-media scopes; nine engine regressions and ten installed-app workflow cases. A real cached CLIP queue index/reference/library/temporal-search/edit/history workflow makes zero model or B2 HTTP requests. Missing-media cases also stay offline, including migration frame preparation. | Media hydration policy and these complete built-in workflows are verified in `ve-ovz.19`. Arbitrary callback networking and explicitly consented remote providers are separate boundaries; input scoping and the remaining owned-service privacy checks are complete in `ve-ovz.14`. |
+| VE-NFR-012: bounded scale degradation, no corruption/unbounded React payloads/full-vector scan each query | Native ANN candidate retrieval, bounded video starts/windows, stable pagination, cache deletion/replacement regressions; clean `c4f1d89` indexes 100k moments in 4,000 bounded batches (320 s including preparation) and pages at most 100 hits per query; consumer deep batches are bounded by source seconds. | Search architecture and the measured 100k workload pass on the current source. Larger-than-100k books degrade through longer indexing only by design; no measurement beyond 100k moments exists. |
+
+Detailed benchmark provenance and raw samples are linked from
+`docs/temporal-search-performance.md`. Reused fixtures inherit original
+indexing metrics; they do not measure a fresh index build. First-query times
+include snapshot loading and are not warm-query latency or cold OS-cache tests.
+
+## Security and privacy
+
+| Section 13 requirement | Evidence inspected | Assessment and follow-up |
+| --- | --- | --- |
+| Opt-in/configurable, pinned, checksum-verified and disableable model downloads | Engine defaults local-only; built-in pinned model inventories carry upstream Git/LFS digests. The verified file resolver checks downloads and cached files before returning bytes or ONNX paths, including external weights. Real transfers, corrupt caches and installed-package rejection are tested. | Checksum hardening and custom commit pinning are integrated in consumer `80ce4f58`. Remote file requests must match the worker repository/revision, and embedding identities isolate revisions. See `docs/model-integrity.md` and `docs/model-revisions.md`. |
+| Scoped provider inputs, local built-in provider, injected network declaration and application consent | Explicit download/inference declarations and separate application grants guard temporal and compatibility dispatch. Actual HTTP tests cover rejected and authorized providers, revocation, changed declarations, and per-Engine scope. CLIP/CLAP declare local inference. | Engine and consumer consent integration are verified. Identical registrations preserve overlapping searches; replacement/removal revokes later calls. Consumer `tests/v2/search-reference-range-scope.test.ts` proves the application's `search_moments` forwards only the prepared reference and the requested `reference_range` window: each window retrieves only its own moment, negative, zero-length and frame-less windows and image references with ranges fail before any search, and unknown reference ids are rejected. |
+| Argument arrays, bounded outputs, timeouts, cancellation and scoped workspaces; no untrusted shell | Shared media-process limits and isolated model pool provide deadlines, process-group cancellation and owned scratch cleanup. Consumer actual queue tests cancel stalled model requests and preserve source bytes/status. | Engine isolation and consumer signal forwarding are implemented and tested. See `docs/media-limits.md` and `docs/model-isolation.md`. |
+| Malformed codec, oversized image, decompression bomb and model OOM fail job without book corruption | Tests reject malformed/oversized/high-expansion inputs; an actual heap-exhausted worker leaves a live engine writable and reopenable and permits fresh-worker retry. Consumer cached-model queue tests verify malformed-image failure and corrected retry. | Process isolation, typed job failures and consumer integration are verified. The worker heap cap is not an OS-wide memory limit. |
+| Excerpts/explanations treated as user content | Consumer `tests/v2/search-hostile-content.e2e.test.ts` seeds a real book whose name, artifact label and indexed description carry script, event-handler and javascript: payloads, serves it through the actual MCP server and Vite app in Chromium, and verifies the library search card header, excerpt, thumbnail alt text, signal chip title, notice text and media library dialog render the literal strings; injected hostile explanation and error responses render literally too. No window marker is set, no active element exists, and no page error occurs. | Complete for every owned search result surface (`vb-3esp`). Media response sandboxing remains separately verified. |
+| Logs contain IDs/hashes/sizes/phases/codes rather than secrets/full content | Consumer `7cd32e85` routes owned runtime console and persistent diagnostics through fixed events and validated UUIDs, queue IDs, counts, enums and error codes. Tool/job names require trusted registration. Tests cover private returned/thrown errors, real queue/provider failures, book reopening, explicit chat history and a real subprocess with multi-megabyte private output. | `vb-wtu9` is complete: 3,160 default tests, 19 model/queue tests including all 3 real cached-model cases, media rollback E2E, lint/types/knip/builds and isolated clean install pass. Full caller/job error details and explicit chat content remain available; old diagnostic files are not rewritten. The remaining owned-service offline/privacy invariants are verified in `ve-ovz.14`. |
+| Content hashes are identity, not authorization | Engine provider tests reject foreign artifact UUIDs and raw hash/path references. Actual HTTP file, manifest, and range tests reject mismatched owning books and raw hashes. All HTTP routes reject foreign Origin/Host before parsing or accessing content. | Consumer `9d0179ad` (`vb-0ujh`) enforces loopback Host/Origin policy and binds to `127.0.0.1`. It remains a local single-user service: native clients and accepted local origins can access all books. Hashes and UUIDs are not credentials; this is not a remote or multi-user authentication design. |
+| Remote publication/backup explicit, never triggered by local search | Actual application B2 HTTP counters stay empty through cached indexing, reference preparation, search, edits and history. Explicit backup then performs HEAD/PUT/verification. A configured local catalog backup target stays absent through indexing/search/history and is written only by explicit backup. | The tested object-store and catalog publication boundaries are verified. This is not an authorization audit of every transport or remote service. Owning-book/API access and the remaining owned-service privacy checks are verified in `ve-ovz.14`. |
+
+Remote hydration integrity is verified in engine `07f0515` and consumer
+`20db91d1` (`ve-ovz.18` / `vb-fuib`). Eight engine HTTP/public-API cases cover
+same-size wrong content, partial failures, concurrent valid/corrupt transfers,
+state preservation, retry, cached reuse, and a forget during download. SHA-256
+verification precedes local publication; a final synchronous tombstone check
+and rename prevent an in-flight download from republishing forgotten bytes.
+The actual application B2 adapter rejects a corrupt HTTP response and supports
+valid retry and cached reads after the fixture server shuts down. Existing local
+objects are not rehashed on every read. This evidence does not establish
+cross-book authorization or the complete offline workflow.
+
+Engine `bf952c5` adds `withLocalMedia`: asynchronous per-book scopes block implicit
+CAS downloads with typed `MEDIA_MISSING`, while concurrent explicit reads and
+other books remain independent. Compatibility and temporal search/indexing,
+edits and history use the policy automatically. Restores succeed for committed
+metadata when unavailable bytes prevent workspace hydration. Consumer `96892e26`
+applies the scope to direct and queued semantic indexing (including migration
+frames), temporary references, library asset projection, and ingest status/retry.
+Auxiliary failures preserve source-media readiness. Nine missing-media app cases
+exercise the actual B2 adapter and explicit retrieval retry; the additional
+cached-model case covers search, edits, restoration and explicit backup.
+
+This completes `ve-ovz.19` / `vb-ld4o`, not the complete security/privacy audit.
+Cold-index preparation in the media picker was fixed separately in consumer
+`185ec77c` (`vb-ackl`), with a real 1,200-vector queue/poll/retry and
+persisted-index regression.
+
+Consumer `64da847a` completes `vb-j3qy`: completed image-subject edits, Library
+copies and Duplicate Asset schedule detached indexing jobs deduplicated by
+artifact and source hash. Semantic indexing and frame preparation select stream
+records matching the current file mapping, rather than an older stream for the
+same path. The Library reports missing current-source CLIP coverage and offers
+an explicit repair action; its MCP endpoint returns queued per-book jobs and the
+client polls their status. Repair skips covered media, prepares missing video
+frames, continues past per-asset failures and preserves source readiness.
+
+Validation includes 3,176 default tests across 371 files, explicit cached-model
+repair of zero-vector and stale-stream cases, corrupt-media continuation and
+idempotence, and a real Chromium Library repair/poll/Similar-search flow. Lint,
+test types, client/server builds, dead-code checks, generated command references
+and isolated clean installation pass. The original failing edited image now
+ranks itself first with no book errors. A saved-library repair resolved all 74
+current-source coverage gaps; the final scan reports zero missing visual
+indexes. One empty copied video was recovered from its exact preceding
+nonempty artifact revision after SHA-256, dimensions and duration verification
+(`vb-v9nq`), then indexed and retrieved successfully. These fixture and local
+library results do not replace the frozen-corpus or reference-hardware gates.
+
+Provider dispatch scoping is hardened in Engine `dffaf21` (`ve-ovz.20`).
+Compatibility preparation and embedding receive a fresh options object containing
+only the supplied cancellation signal and timeout. Actual HTTP fixtures showed
+that the previous implementation transmitted extra application context and let
+providers mutate query controls. Six regressions now verify all four modalities,
+selected file/text transmission, caller-option isolation, rejected foreign-book
+UUIDs and raw hash/path references, and temporal reference/query separation.
+The engine's 347 default tests, types, dead-code checks, build and installed
+package smoke test pass, as does Node 22/24 CI. See
+`docs/search-provider-consent.md` for the trusted in-process provider boundary.
+Consumer `0661c994` vendors the exact verified package (SHA-256
+`cb71bf39f3a088058b44273e402569c0a7c90bbb8abb8866f4118979d5ddfbb8`).
+Its 3,176 default tests, 35 explicit model/offline/repair cases, lint, test
+types, client/server builds, dead-code checks and isolated clean install pass.
+`vb-g3ai` is complete; its application commit was local at that validation point.
+
+The HTTP audit reproduced private fixture disclosure through modern MCP with
+an untrusted Origin and Host. Consumer `9d0179ad` fixes this at both the direct
+MCP handler and Express entry, before CORS, body parsing, static files, or API
+routes. The server binds to `127.0.0.1`, accepts loopback browser origins and
+native clients, and permits the bundled extension's exact stable origin.
+Twenty-three direct-handler cases and actual HTTP/Chromium tests cover foreign,
+opaque and malformed origins; rebinding hosts; all routes including uploads and
+archives; permitted preflight; extension-worker MCP discovery/tool calls; and
+owning-book/hash/range denials. The public extension key pins a development ID,
+not a credential. Consumer README documents the trusted local-client boundary.
+
+A subsequent real browser fixture exposed another path: uploaded SVG scripts
+ran in the media server's origin and read private book names through MCP.
+Consumer `43d31b32` applies CSP sandbox and `nosniff` to media and archive
+responses. The identical private-query fixture no longer executes. Browser
+regressions verify blocked SVG/HTML scripts, forms, and external resource
+requests, preserved inline SVG styling and image rendering, and actual H.264
+playback. The final combined run also verifies the Library's real cached-model
+repair and Similar search flow. All 3,199 default tests across 372 files,
+seven explicit HTTP/Chromium/model E2E cases, lint, test types, dead-code checks,
+client/server builds, and isolated clean installation pass.
+
+`vb-0ujh`, `vb-sby1`, `vb-3esp`, and `vb-bkgq` are complete. Consumer
+`fc751e79` adds the hostile search text browser checks and application provider
+range/input evidence that close `ve-ovz.14`. Its full default suite passed
+3,202 tests across 374 files, with lint, test types, dead-code checks, and both
+builds passing. These local service checks do not establish authentication
+for remote or multi-user deployments.
+
+## Migration, consumer and packaging
+
+Section 15.4 migration is covered by `tests/migration.test.ts`, the pinned v4
+schema fixture and consumer migration tests. Evidence includes valid notebook
+graphs/generation choices, properties/prompts/messages, real timed media,
+empty and large catalogs, content-hash validation, unchanged source bytes,
+identical reruns, cancellation/process interruption and source edits before
+publication. Consumer queue tests cover durable status/cancel, archive/switch
+recovery and idempotent reindex scheduling. `ve-ovz.5`, `ve-ovz.6` and `vb-6eu9`
+are complete. This migration evidence does not prove general edit durability.
+
+Engine `e328f5e` passed 341 default tests (12 opt-in cases skipped), typecheck
+and knip. Runtime `bf952c5` also passed build and standalone installed-package
+checks for local-media denial, explicit retry, checksum rejection, README usage,
+and native image decoding. [Node 22/24 runtime CI](https://github.com/justintanner/videobook-engine/actions/runs/34007728660)
+passed. `e328f5e` adds only the configured-catalog publication regression.
+Consumer `96892e26` passed 3,171 default tests across 370 files (4 opt-in cases
+skipped), 38 final explicit workflow/migration/indexing checks including the real
+cached-model workflow, and the separate 19 model/queue cases with all three
+cached-model cases enabled. Lint, test types, dead-code checks, client/server
+builds and the final isolated worktree clean install passed. The initial full run
+caught the migration-frame hydration bypass while it was being fixed; the final
+full-source run is green.
+
+The earlier model-revision baseline `3f9b37d` passed 324 tests (12 opt-in tests skipped), typecheck, knip,
+build and the 22-group API benchmark smoke. Node 22/24 CI passed tests,
+builds and clean-package verification at `5db34ef`, which differs only by a
+bounded cleanup retry in a merge test. The first Node 24 run failed removing
+that temporary database directory after its assertions passed; no runtime
+source or semantic assertion changed in the cleanup correction. The consent package was also installed
+locally with explicit cached CLIP/CLAP inference enabled. Eleven provider-access
+tests exercise actual HTTP dispatch, denied/changed/revoked consent, identical
+registration during an overlapping query, and compatibility modalities.
+
+Earlier consumer `80ce4f58` passed 3,151 tests across 366 files, lint, test types,
+dead-code checks and both builds. Its three real-model queue cases, skipped by
+default, passed in the explicit 37-test focused run. Those cases include an
+existing-image Similar reference and indexed source retrieval with downloads
+disabled. Corrupt pinned caches fail both actual indexing and reference jobs
+with `MODEL_UNAVAILABLE` while source bytes/status remain unchanged. This
+integration exposed a preparation race: missing tokenizer failure could precede
+a corrupt config check in the other branch. Engine `162bdcd` awaits both
+branches and prioritizes integrity failure; repeated public-provider and
+consumer queue tests verify the correction.
+
+Both worktree and exact committed-source (`80ce4f58`) clean installs passed
+native Sharp, MCP create/list, client delivery and graceful shutdown without a
+sibling engine checkout.
+
+Consumer `b4e369b5` vendors `videobook-engine-5.3.2-c4f1d89.tgz`, SHA-256
+`79f42f492fa569d13993f433da44a3b1ee11a4536fb54246d825d61978284c54`.
+Its provenance file records the exact engine revision; application and engine
+resolve one DoltLite 0.50.6 and one Sharp 0.35.4. The application remains 0.1.0.
+Local package smoke and
+vendored-consumer verification do not prove installation of a published
+registry package: `ve-yc7` and `ve-orp` retain that release gate. Registry
+authentication was rechecked on September 6, 2026 using the environment's
+`NPM_TOKEN` explicitly through an isolated temporary npm configuration.
+Both npm authentication and a direct registry authentication request returned
+401. Publishing the packed 5.3.2 candidate from `4b85f01` then failed with
+`E404` on the registry PUT, reporting that the package was unavailable or the
+token lacked access. No package version or release tag was created. The
+temporary authentication configuration was removed. The obsolete 0.1.0 and
+2.0.0 publication tickets are superseded by
+`ve-yc7`; they are not additional release targets.
+
+Subsequent checksum hardening adds verified file resolution for all built-in
+model paths, complete upstream digest inventories for three pinned snapshots,
+cache-staging cleanup and public-provider corruption regressions. Nine explicit
+real-model tests pass, including MiniLM external ONNX weights and offline reopen.
+The installed package rejects corrupt pinned configuration and runs cached
+CLIP/CLAP inference. The consumer now includes this verified resolver and
+explicit application-owned provider download consent.
+
+All 12 explicit real-model checks passed at model-revision baseline `3f9b37d`. The custom-revision
+suite serves actual cached CLIP/CLAP/MiniLM files under custom repository IDs,
+verifies fixed revision URLs and integrity receipts, indexes and queries real
+media, reopens offline, rejects reuse under a different revision, and loads
+explicit local model directories. Sixteen default contract tests cover missing
+or moving revisions, configuration snapshots, repository-name collisions,
+worker file scope, separate modality defaults and legacy visual cache safety.
+
+Legacy compatibility custom image models previously shared the built-in CLIP
+vector identity without recording their provenance. The compatibility visual
+space now uses `compat-visual-v2`, requiring a one-time reindex instead of
+reusing ambiguous old vectors. Temporal index identities, including those used
+by the media library, are unchanged. Source files remain intact. This migration
+and the commit requirement for custom remote models are documented in
+`docs/model-revisions.md`; `ve-ovz.15` and consumer `vb-3f45` are complete.
+
+The NFR measurement pass (`ve-ovz.9`) records current-source distributions
+for VE-NFR-001/002/004/005/006/007/009/012 and the actual model-pipeline
+cadence for VE-NFR-003 in this document, `docs/edit-performance.md` and
+`docs/temporal-search-performance.md`. Every measured gate passes on the
+available M1 Pro; the M2 Pro reference device named by the PRD has not been
+measured and M1 Pro results are recorded as M1 Pro results only. The full
+quality corpus (`ve-s84`) and published-package verification (`ve-yc7`, `ve-orp`)
+remain outstanding. Exact reference-device acceptance (`ve-ovz.22`) is deferred
+because the hardware is unavailable; full E4/E5/MVP acceptance is not claimed.

package/docs/search-provider-consent.md

@@ -0,0 +1,67 @@
+# Search provider network consent
+
+Injected search providers must declare `networkAccess` with two explicit booleans:
+
+- `modelDownloads`: preparation or execution may fetch model files.
+- `inference`: execution may send query text or selected media to a remote service.
+
+The declaration describes capability. It does not grant permission. The application supplies a separate `SearchProviderConsent`. Each permission defaults to denied and only literal `true` grants it. Download consent never authorizes remote inference.
+
+```typescript
+const provider = new MySearchProvider({
+  networkAccess: { modelDownloads: false, inference: true },
+});
+engine.temporalSearch.providers.register(provider, { inference: true });
+```
+
+The example assumes the application's configuration or user interaction has authorized remote inference. The engine does not display consent UI or infer authorization from a model manifest, provider name, or the fact that a provider was passed to it.
+
+A local injected provider declares `{ modelDownloads: false, inference: false }` and needs no grant. The built-in temporal CLIP and CLAP providers declare local inference and model downloads according to their constructor's `allowModelDownload` option. Constructor options are copied and frozen. Explicit online preparation outside an Engine remains controlled by that constructor option. When registering a download-enabled built-in with an Engine, also pass `{ modelDownloads: true }`. For normal offline use, omit `allowModelDownload` and register without consent.
+
+## Registration and revocation
+
+Temporal registration validates declarations and grants before accepting a provider. Missing or malformed declarations produce `INVALID_INPUT`; denied network permissions produce `OFFLINE`. A failed replacement leaves the existing registration intact.
+
+Authorization is checked before every `prepare` and embedding invocation, including after an awaited preparation step. Grants and declarations are snapshotted. Changing the declaration requires registration again; mutating a previously supplied consent object does not change an existing grant. Re-register with the desired grant, or use:
+
+```typescript
+engine.temporalSearch.providers.unregister(provider.manifestId);
+```
+
+Registering the same provider with unchanged capabilities and grants is idempotent, so repeated registration does not interrupt overlapping searches. Removal or successful replacement revokes future calls through the old registration, including the embedding stage of a query currently awaiting preparation. It cannot recall content already delivered to a running provider. Registrations and grants belong to one Engine instance and are not persisted in semantic data, history, or action logs. Reopening an Engine requires registration again.
+
+## Compatibility similarity API
+
+`EngineConfig.similarity.provider`, `.audio.provider`, and `.text.provider` use the same declarations and dispatch checks. Set `providerConsent` next to each injected provider:
+
+```typescript
+const engine = createEngine({
+  rootDir,
+  similarity: {
+    provider: visualProvider,
+    providerConsent: { inference: true },
+    text: {
+      provider: textProvider,
+      providerConsent: { inference: true },
+    },
+  },
+});
+```
+
+These configurations are checked when the provider is invoked, returning the normal `Result` error from preparation, indexing, or query operations. Each modality owns its grant; audio and text do not inherit the visual provider's consent. Recreate the Engine to change a compatibility provider's grant. Built-in compatibility providers retain their existing `allowModelDownload` configuration and local inference behavior; `providerConsent` applies only to injected implementations.
+
+The new properties are optional in TypeScript so existing integrations still compile on this patch version. **Undeclared injected providers fail at runtime.** Update them to declare their capabilities explicitly. This is an intentional fail-closed migration, not an inference that old providers are local.
+
+## Input and trust boundary
+
+Temporal text dispatch supplies only the query string; preparation receives no search payload. Neither invocation receives the Engine, catalog, book metadata, candidate IDs or other artifacts. Pure reference queries operate on stored/prepared vectors without calling an embedding provider.
+
+Compatibility dispatch resolves the artifact in the selected Engine before choosing its CAS media path or normalized text. A foreign-book artifact UUID, raw object hash or filesystem path cannot substitute for that artifact reference, even when the source bytes exist locally. Labels, metadata, sidecars and unrelated artifacts are not passed to the provider.
+
+Each compatibility preparation and embedding invocation receives a fresh `MediaOperationOptions` object containing only supplied `signal` and `timeoutMs` controls. Query/index controls (`kind`, `force`, `limit`, `minScore`) and extra runtime application context remain inside the engine. Provider mutation of its options copy cannot alter the caller's query controls, and preparation cannot add fields to the later embedding options. The original cancellation signal is preserved.
+
+Compatibility indexing authorizes its entire selected media source; it does not accept a time range. Direct application indexing and reference preparation still choose the media paths and ranges supplied to built-ins. Application range selection and HTTP access authorization require their own checks.
+
+A declaration is a contract with trusted injected JavaScript, not a network sandbox. Providers can lie about capabilities, access their own ambient credentials, or inspect files accessible to the host process. Applications must trust and audit injected implementations. Consent gates prevent the engine from dispatching to undeclared, unauthorized, changed, or revoked providers; they do not intercept arbitrary socket or filesystem calls made by malicious code. These Engine dispatch checks do not prove HTTP/API authorization, filesystem isolation against malicious providers, or whole-application input/range scoping; those remain separate boundaries.
+
+`tests/search-provider-access.test.ts` exercises actual Engine dispatch against an HTTP receiver, including rejected grants with zero requests, query-only transmission, frozen providers, Engine isolation, revocation/replacement during preparation, capability mutation, built-in declarations, compatibility image/video/audio/text denial and separate grants, exact selected-file/text transmission, rejected cross-book/path/hash references, copied operation controls and provider-mutation isolation. The HTTP fixtures explicitly carry private adjacent files, metadata and runtime context so unintended transmission fails the regression.

package/docs/semantic-durability.md

@@ -0,0 +1,55 @@
+# Semantic commit recovery
+
+Semantic mutations and their outbox entry commit in one SQL transaction.
+The engine then stages each declared changed semantic table, creates one Dolt
+revision carrying the operation ID, and removes the outbox entry in a second
+SQL transaction. Reopening replays pending outbox entries before exposing the
+catalog.
+
+For a mutation that changes rows, recovery uses the remaining working diff to
+finish the original commit. If that commit is already durable, no semantic
+diff remains and recovery only clears the outbox. Provenance-only operations
+intentionally create empty commits, so absence of a diff is ambiguous. During
+recovery, the engine searches recorded operation IDs before creating another
+empty commit. This includes an earlier committed operation followed by a later
+successful write; checking only HEAD would duplicate the older entry.
+After a failed semantic write, the next semantic write first recovers pending
+outbox work. This preserves operation order and prevents a later mutation
+from absorbing an earlier transaction's uncommitted history. Ordinary successful
+writes do not add an outbox scan to their path.
+
+`tests/semantic-crash.test.ts` launches separate Node processes against copies
+of a closed fixture. It first records the actual boundary sequence for a
+four-operation edit affecting clips and transforms and for a provenance-only
+operation. It then sends the executing process SIGKILL at every observed
+boundary, including every individual table-staging call. The callback writes
+its boundary synchronously before termination; the parent checks both the
+last boundary and the SIGKILL exit signal. A timeout cannot masquerade as a
+successful boundary test.
+
+The matrix covers:
+
+- Semantic changes inside the SQL transaction, before outbox insertion.
+- Outbox insertion before the semantic SQL commit, and immediately after it.
+- Each table stage, before the Dolt commit, and immediately after it.
+- Before outbox deletion, after deletion before its SQL commit, and after
+  that commit.
+- Repeated termination during recovery of an already committed SQL batch.
+- A committed provenance operation left in the outbox while another operation
+  succeeds, followed by process termination.
+
+Before the semantic SQL commit, reopen must show the original state and no
+new operation. After it, reopen must show all three clips, the transform and
+exactly one corresponding history entry, or exactly one provenance entry.
+The tests also verify the edit audit, empty outbox, no staged rows, subsequent
+successful writes and repeated reopen. No graceful engine close runs in the
+killed process. Existing exception-recovery tests remain useful for error
+responses, but do not substitute for this matrix.
+
+These tests verify application-visible transaction boundaries in the native
+catalog on the executing platform. They do not simulate physical disk failure
+or interrupt the interior of a native SQL statement. Migration publication
+and consumer archive switching have separate process-interruption fixtures.
+The optional `semanticCommitBoundary` callback is a diagnostic fault-injection
+hook; normal consumers should omit it. `after-table-stage` may occur several
+times for one operation, and recovery can invoke callbacks during engine open.