@arnilo/prism 0.5.6 → 0.7.0

package/docs/compaction-observational-memory.md

@@ -4,10 +4,12 @@
 
 `@arnilo/prism-memory/compaction/observational-memory` is an optional subpath for source-backed observational memory and fast compaction.
 
-Current status: ledger/projection/render/recall utilities, explicit worker runtime, fast compaction strategy, inert extension helper, recall tool, and status/view command factories are available.
+Current status: ledger/projection/render/recall utilities, optional work-scope indexing, explicit worker runtime, fast compaction strategy, inert extension helper, recall tool, and status/view command factories are available.
 
 This package is distinct from `@arnilo/prism-memory` working/semantic memory: observational memory compresses and recalls source-backed observations/reflections; semantic memory retrieves embeddings; working memory stores the current structured profile/state. Hosts may compose both.
 
+This page's memory stays **episodic**: the ledger records what happened in a session, its observer/reflector/dropper workers are the only writers of observations and reflections, and nothing downstream re-observes the transcript. Typed notes that want to outlive the session (facts, procedures, file references) are a separate layer — the [memory fabric](memory-fabric.md) — which views an observation by id as an `episode` note without copying it, never wraps or replaces these workers, and keeps its own recall path. Promotion out of the ledger is an explicit host write, not a side effect of compaction.
+
 ## Four-layer provider context
 
 Observational memory composes four independent layers for long sessions (Mastra-style):
@@ -19,7 +21,9 @@ Observational memory composes four independent layers for long sessions (Mastra-
 | **Reflections** | Higher-level summaries over observation ids | Reflector worker on observations after last reflection coverage when `reflection.observationTokens` met |
 | **Raw-source retrieval** | Exact branch messages behind a memory id or cursor page | `recallObservationalMemory()` / `recallObservationalMemoryBranchPage()` / `createRecallMemoryTool()` — exact-id or cursor paging only; no semantic search |
 
-Activation is explicit: `createObservationalMemory().attach()` coordinates post-run observe/reflect/drop and `context.compactAfterTokens` compaction. Import and extension `setup` start nothing. Recall, commands, and utilities fail closed on invalid ids, wrong `sessionId`, ambiguous tool input, or oversized pages. Pass `secrets` for exact-value redaction in render/recall/worker paths. Branch isolation: hosts supply current-branch `appendEntry` and `getEntries`; mismatched store/session pairs fail closed after append.
+The opt-in work-scope index filters the observation and reflection layers for a host-selected working set. It is not a fifth context layer, retrieval system, or session scope.
+
+Activation is explicit: `createObservationalMemory().attach()` coordinates post-run observe/reflect/drop and compaction — by default `context.compactAfterTokens`, or whatever host gate `trigger` / `shouldCompact` supplies. Import and extension `setup` start nothing. Recall, commands, and utilities fail closed on invalid ids, wrong `sessionId`, ambiguous tool input, or oversized pages. Pass `secrets` for exact-value redaction in render/recall/worker paths. Branch isolation: hosts supply current-branch `appendEntry` and `getEntries`; mismatched store/session pairs fail closed after append.
 
 See `examples/observational-memory-lifecycle.ts` for attach → turn → projection/recall/page without live credentials.
 
@@ -31,6 +35,8 @@ Use `createObservationalMemoryCompactionStrategy()` when compaction should rende
 
 ## Inputs / request
 
+**Option surfaces** — `appendEntry` takes `ObservationalMemoryAppendOptions` (custom observation/reflection text, trust, and metadata); `createWorkScopeController` takes `WorkScopeControllerOptions` (session, `appendEntry`, optional `secrets`).
+
 Memory records use `SessionEntry.kind: "custom"` with `entry.data.type` markers:
 
 | Type | Payload |
@@ -38,6 +44,9 @@ Memory records use `SessionEntry.kind: "custom"` with `entry.data.type` markers:
 | `om.observations.recorded` | `{ observations, coversUpToId? }` — successful observer runs append coverage even when `observations` is empty. |
 | `om.reflections.recorded` | `{ reflections, coversUpToId? }` |
 | `om.observations.dropped` | `{ observationIds, coversUpToId? }` |
+| `om.scope.opened` | `{ id, parentId?, kind?, label? }` — host-defined scope tree node. |
+| `om.scope.closed` / `om.scope.entered` / `om.scope.left` | `{ scopeId }` for close/enter; `{}` for leave. |
+| `om.scope.bound` / `om.scope.unbound` | `{ scopeId, refs }` — many-to-many `om:<12-hex>` or `reflection:<12-hex>` membership. |
 | `om.folded` | Compaction `data.memory` folded details. |
 
 Ids are known, source-backed 12-character lowercase hex strings matching `^[a-f0-9]{12}$`.
@@ -67,18 +76,21 @@ Key exports:
 | Export | Purpose |
 | --- | --- |
 | `foldObservationalMemoryLedger()` | Fold custom memory entries into observations, reflections, drops, and coverage markers. |
+| `foldWorkScopeMap()` / `createWorkScopeController()` | Fold the opt-in scope index or append validated open/close/enter/leave/bind/unbind entries. |
+| `projectWorkMemory()` | Filter the folded observations/reflections through a scope query; exact-id recall stays unfiltered. |
+| `withWorkScope()` | Open a missing scope, enter it for an async callback, and always leave without closing it. |
 | `isEligibleObservationSourceEntry()` / `eligibleObservationSources()` | Select user/assistant/tool `message` entries for observer input. |
 | `unscannedEntries()` / `observationsUncoveredByReflection()` | Dual coverage helpers for observation scan and reflection windows. |
-| `buildObservationalMemoryProjection()` | Build active/full/folded projections from current branch entries. |
-| `buildObservationalMemoryContextBlocks()` | Render observational-memory + recent-messages context blocks for provider input. |
+| `buildObservationalMemoryProjection()` | Build active/full/folded projections from current branch entries. Optional `invalidatedIds` drops observations whose id or `sourceEntryIds` match, and reflections that rest on them. Full ledger stays for audit. |
+| `buildObservationalMemoryContextBlocks()` | Render observational-memory + recent-messages context blocks for provider input. Same `invalidatedIds` option. |
 | `selectRecentMessageEntries()` / `renderRecentMessageWindow()` | Bounded exact recent-message suffix; count via `keepRecentEntries`, optional token trim via `estimateEntryTokens`. |
 | `createFoldedMemoryDetails()` | Create JSON details for compaction `data.memory`. |
 | `renderObservationalMemory()` | Render reflections and observations into a prepared memory summary. |
-| `recallObservationalMemory()` | Recover source evidence for a known observation/reflection id from supplied current-branch entries. |
+| `recallObservationalMemory()` | Recover source evidence for a known observation/reflection id from supplied current-branch entries. `invalidatedIds` withholds content (`reason: "revoked"`) without injecting derived text. |
 | `recallObservationalMemoryBranchPage()` | Page eligible user/assistant/tool messages around a cursor entry id (`forward`/`backward`, optional `detail: summary|full`). |
 | `createMemoryId()` / `isMemoryId()` | Create/check 12-character ids. |
 | `resolveObservationalMemorySettings()` | Merge `observational-memory` settings with defaults and overrides. |
-| `createObservationalMemory()` / `attach()` | One activation wires post-run observe/reflect/drop and `compactAfterTokens` compaction; returns proxied session, runtime, context provider, and strategy. |
+| `createObservationalMemory()` / `attach()` | One activation wires post-run observe/reflect/drop and compaction (`compactAfterTokens`, or a host `trigger` / `shouldCompact`); returns proxied session, runtime, context provider, and strategy. |
 | `createObservationalMemoryRuntime()` | Low-level explicit flush for advanced hosts or tests. |
 | `createObservationalMemoryCompactionStrategy()` | Render existing folded memory as a standard Prism compaction summary with `data.memory`. |
 | `createObservationalMemoryExtension()` | Inert extension helper that registers the strategy contribution unless disabled. |
@@ -86,7 +98,32 @@ Key exports:
 | `createMemoryStatusCommand()` / `createMemoryViewCommand()` | Optional `om:status` and `om:view` command factories. |
 | `createObservationalMemoryCommands()` | Convenience factory returning status and view commands. |
 
-Pure utilities create no events, workers, tools, commands, credentials, or provider requests. `createObservationalMemoryExtension()` and import alone start nothing. `createObservationalMemory().attach()` runs workers only after proxied `run`/`prompt`/`stream`/`compact` complete (or after `wrapResumeRun` / `wrapResumeStream`). `createObservationalMemoryRuntime().flush()` remains for manual/advanced use. Attached `contextProvider` renders two blocks each turn: `observational-memory` (active reflections/observations aligned to the recent-message boundary) and `recent-messages` (last `keepRecentEntries` message entries in branch order, optionally trimmed by `recentMessageMaxTokens` using `estimateEntryTokens`; oldest dropped first). Compaction uses the same `keepRecentEntries` setting. Observer input includes only eligible `message` entries (`user`, `assistant`, `tool`); memory/compaction/bookkeeping entries advance `coversUpToId` scan coverage without entering the observer prompt. Successful observer/reflector runs append coverage markers even when they record zero facts. Reflection uses only active observations recorded after the last `om.reflections.recorded` entry unless `flush({ fullReflectionRebuild: true })`. Attached `flush()` skips with `run_active` while a proxied run is in flight. The compaction strategy is O(n) over supplied entries and makes no provider call. Tool and command factories are inert until a host registers/selects them.
+Pure utilities create no events, workers, tools, commands, credentials, or provider requests. `createObservationalMemoryExtension()` and import alone start nothing. `createObservationalMemory().attach()` runs workers only after proxied `run`/`prompt`/`stream`/`compact` complete (or after `wrapResumeRun` / `wrapResumeStream`). `createObservationalMemoryRuntime().flush()` remains for manual/advanced use. Attached `contextProvider` renders two blocks each turn: `observational-memory` (active reflections/observations aligned to the recent-message boundary) and `recent-messages` (last `keepRecentEntries` message entries in branch order, optionally trimmed by `recentMessageMaxTokens` using `estimateEntryTokens`; oldest dropped first). Compaction uses the same `keepRecentEntries` setting. Observer input includes only eligible `message` entries (`user`, `assistant`, `tool`); memory/compaction/bookkeeping entries advance `coversUpToId` scan coverage without entering the observer prompt. Successful observer/reflector runs append coverage markers even when they record zero facts. Reflection uses only active observations recorded after the last `om.reflections.recorded` entry unless `flush({ fullReflectionRebuild: true })`. Attached `flush()` skips with `run_active` while a proxied run is in flight. The compaction strategy is O(n) over supplied entries and makes no provider call.
+
+### Work-scope index (opt-in)
+
+`WorkScope` is a host-named, append-only index over one observational-memory ledger. Without `om.scope.*` entries, the map has only its implicit `session` root, context renders the existing active pool, and the dropper keeps its existing behavior.
+
+Use `createWorkScopeController({ session, appendEntry, secrets? })` to `open`, `close`, `enter`, `leave`, `bind`, or `unbind` scopes. Scope ids are host-defined (`[A-Za-z0-9._:/-]{1,128}`, no `..`); there are caps of 256 scopes, depth/stack 8, 4,096 binds per scope, and 512 characters for labels or kinds. Invalid ids, missing/closed parents, duplicate scopes, unknown record ids, and ownership mismatch fail closed. Labels and kinds receive the same secret redaction as observational-memory text.
+
+`projectWorkMemory(ledger, map, { from, include, closed?, kinds? })` returns a filtered observation/reflection view plus outline. `include` is `self`, `self+ancestors`, `self+descendants`, or `lineage`; `closed: "hide"` is the default, except closed ancestors of `from` remain available. Default attached context uses the current leaf with `self+ancestors`, rendering Scope Outline, Reflections, then Observations. The compaction summary — the layer the next run's pack starts from — renders the same projection, so the full ledger never rides into the prefix; the folded payload keeps every observation, so entering another scope can still surface what that summary hid. `recallObservationalMemory()` still reads the complete current branch by exact id.
+
+After a flush records new observations or reflections, it binds those ids once to the current leaf scope only. A host promotes relevant memory explicitly by binding it to an ancestor; a reflection whose bind sits on a **closed** scope can also graduate into durable semantic memory through the fabric's `remember({ kind: "fact" | "procedure", reflectionId })`. While any host scope exists, the runtime skips the observation dropper; the folded-payload byte cap remains a storage safety cap, not working-set garbage collection. `withWorkScope(controller, spec, fn)` opens `spec` if needed, enters it, runs `fn`, and leaves in `finally`; it never closes a scope. This index does not provide resource-scoped observational memory or budget-based dropping as a working-set mechanism.
+
+### Compact-when override
+
+`createObservationalMemory()` accepts a compact-when gate beside the settings: `trigger` (the same union `CompactionOptions.trigger` uses) or the `shouldCompact(context)` shorthand. When either is set it **replaces** `context.compactAfterTokens`; omitted, the token gate is unchanged.
+
+```ts
+const om = createObservationalMemory({
+  observation: { provider, model },
+  shouldCompact: (context) => context.entryCount > 40 || context.estimatedInputTokens / context.inputCapTokens >= 0.9,
+});
+```
+
+- `context.entryCount` counts current-branch entries; `context.estimatedInputTokens` is this package's own `estimateEntryTokens` sum; `context.inputCapTokens` comes from `resolveInputCap` on `attach({ sessionModel })`.
+- Both token numbers resolve lazily, so a callback that only reads counts works with a `sessionModel` that declares no `contextWindow`. Reading the cap without one throws inside the callback, and the gate then decides **false** — the sync loop reports it through the `debug` sink (`observational-memory:compaction-trigger-error`) and never compacts on a guess. `input_ratio` needs a resolvable cap and throws instead.
+- An unknown trigger `type` or a non-function `shouldCompact` throws at `createObservationalMemory()`, before any session work. Tool and command factories are inert until a host registers/selects them.
 
 ## Request/response example
 
@@ -105,8 +142,13 @@ import {
   createObservationalMemoryCommands,
   createObservationalMemoryRuntime,
   createRecallMemoryTool,
+  createWorkScopeController,
+  foldObservationalMemoryLedger,
+  foldWorkScopeMap,
+  projectWorkMemory,
   recallObservationalMemory,
   renderObservationalMemory,
+  withWorkScope,
 } from "@arnilo/prism-memory/compaction/observational-memory";
 
 const om = createObservationalMemory({
@@ -119,11 +161,19 @@ const attached = om.attach(session, {
   appendEntry: (entry, options) => store.append(entry, options),
   sessionModel: agent.config.model,
 });
-await attached.session.run("Continue from prior work");
+const scopes = createWorkScopeController({ session: attached.session, appendEntry: (entry, options) => store.append(entry, options) });
+await scopes.open({ id: "plan:memory", kind: "plan", label: "Memory work" });
+await withWorkScope(scopes, { id: "task:cleanup", parentId: "plan:memory", kind: "task" }, () =>
+  attached.session.run("Continue from prior work"),
+);
 
 const entries = await session.entries();
 const projection = buildObservationalMemoryProjection(entries);
-const summary = renderObservationalMemory(projection.reflections, projection.observations);
+const scoped = projectWorkMemory(foldObservationalMemoryLedger(entries), foldWorkScopeMap(entries), {
+  from: "task:cleanup",
+  include: "self+ancestors",
+});
+const summary = renderObservationalMemory(scoped.reflections, scoped.observations, { outline: scoped.outline });
 const evidence = recallObservationalMemory(entries, "aaaaaaaaaaaa");
 
 const memory = createObservationalMemoryRuntime({
@@ -231,11 +281,14 @@ Ownership: funnel only within the `OwnershipScope` already on the parent agent/s
 ## Related APIs
 
 - [Use-case model selection](use-case-model-selection.md): session vs worker model binding and `resolveUseCaseModel`.
+- [Attention compiler](attention-compiler.md): opt-in per-turn shrink that runs before compaction is considered and resolves the same input cap for `input_ratio` triggers.
 - [Thinking and reasoning](thinking-and-reasoning.md): `thinkingLevel` → provider `compat`.
 - [Provider request policies](provider-request-policies.md): derived `om:{session.id}` on worker generate.
 - [Compaction and retry policies](compaction-and-retry.md): replaceable compaction strategy boundary.
+- [Workflows](workflows.md): a host may use a workflow `nodeId` as a scope id with `withWorkScope`; the workflow runner does not enter scopes itself.
 - [LLM compaction package](compaction-llm.md): existing optional compaction-package pattern.
 - [Session stores and branching](session-stores-and-branching.md): branch entries that observational memory reads and appends to.
+- [Memory fabric](memory-fabric.md): optional typed notes over the same stores; `episode` notes are views of these observation ids, and the ledger stays episodic.
 - [Supervisor delegation](supervisors.md): child sessions whose messages this page's opt-in funnel can copy onto a workspace branch.
 - [Extensions](extensions.md): inert registration pattern for optional package contributions.
 - [Tools](tools.md): host activation and dispatch for optional recall tool contributions.

package/docs/context-and-skills.md

@@ -128,7 +128,7 @@ Each active skill contributes two things the runtime wires together:
 - `Skill` prompt text → rendered as system messages by `skillMessages()` / `skillPromptText()` (active set only). Default `skillsDisclosure: "progressive"` sends `Skill <name>: <description>`; full `instructions` appear only when the skill is in the session `LoadedSkillSet` or disclosure is `"eager"`.
 - `Skill.context: ContextProvider[]` → collected across active skills (`activeSkills.flatMap(s => s.context ?? [])`), resolved through the existing `resolveContextProviders(...)`, and merged into the request's `context` **after** host `AgentConfig.context` blocks. Inactive skills contribute neither instructions nor context.
 
-`toolNames` enforcement is live: because selection routes through `resolveActiveSkills()`, a skill demanding a host-inactive tool throws with `Skill ${name} requires inactive tool: ${missing}` **before the first provider turn** — no provider call, no store write, no partial side effect. This is the fail-fast contract the docs already claimed; the runtime now honors it.
+`toolNames` enforcement is live: because selection routes through `resolveActiveSkills()`, a skill demanding a host-inactive tool throws with `Skill ${name} requires inactive tool: ${missing}` **before the first provider turn** — no provider call, no store write, no partial side effect. When `RunOptions.toolNames` narrows the run, that snapshot is the host-active list — a skill cannot require a registered tool the run did not grant.
 
 ```ts
 import { createAgent, createSkillRegistry, type ContextProvider } from "@arnilo/prism";
@@ -219,7 +219,7 @@ Under pressure on a skill with a loaded body, eviction may demote to catalog-onl
 
 ### Optional tool-result fold
 
-`toolResultFold` on `AgentConfig` / `RunOptions` (run wins) is **off** unless the host supplies a `summarize` callback. When enabled, aged large tool-result messages in the **provider view** become a one-line header plus bounded summary text; session store entries stay raw. Defaults: `minAgeTurns` **2**, `minBytes` **4096**, `maxSummaryBytes` **512** (hard **4096**). Summarizer failure keeps the raw tool result (fail closed). Not a second memory system — use observational memory / compaction for durable recall.
+`toolResultFold` on `AgentConfig` / `RunOptions` (run wins) is **off** unless the host supplies a `summarize` callback. When enabled, aged large tool-result messages in the **provider view** become a one-line header plus bounded summary text; session store entries stay raw. Defaults: `minAgeTurns` **2**, `minBytes` **4096**, `maxSummaryBytes` **512** (hard **4096**). Summarizer failure keeps the raw tool result (fail closed). Not a second memory system — use observational memory / compaction for durable recall, and remember that observational memory stays the source-backed **episodic** ledger of the session: typed, time-bounded notes that outlive a session are a separate layer ([memory fabric](memory-fabric.md)) whose provider arrives through this same inert seam.
 
 ```ts
 await session.run("…", {
@@ -267,8 +267,10 @@ Use `activateAllCapabilities: true` only as a temporary all-skills/all-tools com
 
 - [Agent/session runtime](agent-session-runtime.md): consumes host-selected context providers and skills from explicit agent config.
 - [Input and prompt assembly](input-and-prompt-assembly.md): default prompt builder and provider-input assembly helper.
+- [Attention compiler](attention-compiler.md): opt-in ratio gate that rewrites aged history and tool results for one request, leaving resolved context blocks and skills in place.
 - [Instruction injection](instruction-injection.md): package injectors contribute `contextBlocks` that merge after host+skill provider blocks.
 - [Retrieval-augmented generation](rag.md): optional retrieved citations contribute through the same explicit inert context seam.
+- [Memory fabric](memory-fabric.md): optional typed notes whose provider contributes the same `working-memory` / `semantic-memory` blocks this seam already carries, under a host-registered name.
 - [Public contracts](public-contracts.md): `ContextProvider`, `ContextResolutionContext`, `ContextBlock`, `Skill`, `SkillRegistry`, `PromptBuilder`, and `PromptBuildRequest`.
 - [Middleware hooks](middleware-hooks.md): `context` and `prompt_build` hooks.
 - [Contribution registries](contribution-registries.md): inert context provider and skill contributions.

package/docs/contributing.md

@@ -0,0 +1,37 @@
+# Contribution quality budgets
+
+## What it does
+
+Records the code-quality ceilings a change must not raise, and the procedure for lowering them. Three budgets exist today: the non-null assertion allowance per directory, the public export surface per package, and the benchmark/timing envelopes. All live in `scripts/budgets.json` and are enforced by in-chain gates that run as part of `npm test`.
+
+## When to use it
+
+Before a change that adds a `!` non-null assertion, adds a public export, or moves code between directories; and after any sweep that removes them — the recorded numbers are lowered in the same change, never later.
+
+## Non-null assertions
+
+`style/noNonNullAssertion` is an **error** repo-wide in `biome.json`. Directories that still carry legacy sites are switched back to `off` per directory through `overrides`, and `scripts/budgets.json` → `nonNullAssertions` records how many sites remain in each of them.
+
+Rules:
+
+- New code does not add `!`. Narrow the value once — a local `const` behind an explicit guard — or capture the seam in a helper (the durable-session and elicitation seams are the models). Do not trade the assertion for an `as` cast or a `??` placeholder.
+- Keep the allowlist and the budget rows identical: the gate asserts that the `biome.json` allowlist directories and the `nonNullAssertions.byPath` keys describe the same set, so there is no ambiguity about which row an override corresponds to.
+- A sweep lowers the directory's `byPath` number and the `ceiling` in the same change. Raising a number needs a reason in the `$comment`.
+- When a directory reaches zero sites, delete its `overrides` entry and its `byPath` row: the gate fails on a stale zero row rather than letting the allowance rot.
+- The clusters swept in that pass — `src/agent-approval.ts`, `src/agent-loops.ts`, `src/agent-tool-dispatch.ts`, `packages/prism-coding-tools/src/agent/{glob-match,delete,git*}.ts`, `packages/prism-coding-tools/src/agent/language/framing.ts`, and `packages/prism-coding-tools/src/security/{sandbox-tar,sandbox-fs-operations}.ts` — are re-enabled as errors by the last override in `biome.json` (the last matching override wins), so they cannot regress. The gate rejects a rename that would silently drop that enforcement.
+- Measure locally with one pass: `node_modules/.bin/biome lint --only=style/noNonNullAssertion --reporter=json .`. `--only` reports the rule inside allowlisted directories too, which is what makes the counting gate possible; the gate also fails when a site appears outside the allowlist.
+
+## Public export surface
+
+`scripts/budgets.json` → `exportCounts` records a ceiling per package. Growth fails the gate and names the package and the exact delta. Prefer re-exporting an existing symbol (or documenting the host-side composition) over widening a package's surface; moving an existing internal helper between modules does not change the count, because the counter dedupes by name.
+
+## Timing envelopes
+
+Benchmark medians and p95 ceilings in `scripts/budgets.json` are non-flaky sanity bounds, not portable SLOs; the startup gate additionally compares a machine-relative ratio so external CPU load cannot fail the suite. `scripts/benchmark.mjs` produces the evidence-of-record numbers.
+
+## Related APIs
+
+- `scripts/budget-gates.mjs`: `measureNonNullAssertions()`, `evaluateNonNullBudget()`, `measureExportCounts()`, `checkExportBudget()`, and the startup helpers.
+- `scripts/budget-gate.test.mjs`: the in-chain gate, including the negative fixtures that prove each failure mode.
+- `scripts/run-all-tests.mjs`: the stages `npm test` runs (`gate suites` includes the budget gate).
+- [Release and install](release-and-install.md): the release gates that re-assert these budgets before publication.

package/docs/conversations.md

@@ -117,7 +117,7 @@ Behavior notes:
 - `create` with an explicit `id` writes with `expectedVersion: 0` (create-only): a duplicate create never overwrites the winner's metadata and returns the existing thread.
 - `branch()` enforces `maxActiveBranches` on its read snapshot; the version guard inside the same write makes the cap exact even under concurrency (a concurrent branch cannot slip past the cap), and the marker keeps branch refs append-only.
 - `archive()` on an already-archived thread is a no-op; a stale `branch`/`archive` racing a delete fails `not_found` (the row is gone) and the write path never re-creates a deleted thread — delete wins.
-- Deletion purges the whole session ledger (entries, runs, events, tool calls, usage, branches, search rows) through `lifecycle.applyRetention`; legal holds block deletion and report `held: true`.
+- Deletion purges the whole session ledger (entries, runs, events, tool calls, usage, branches, search rows) through `lifecycle.applyRetention`; legal holds block deletion and report `held: true`. Semantic-memory `forget({ hold: true })` is a separate knowledge-store hold (see [Working and semantic memory](working-and-semantic-memory.md)); conversation delete does not walk embeddings.
 
 ## Security and performance notes
 

package/docs/core.md

@@ -21,6 +21,8 @@ npm install pg
 npm install @nats-io/jetstream @nats-io/transport-node
 ```
 
+Every peer below is optional and fails closed at first use; the [optional peer dependencies](peer-dependencies.md) matrix lists the exact ranges, pins, and which of them reach the network.
+
 ## Subpaths Map
 
 | Subpath | Description | Optional Peers |

package/docs/dev-inspector.md

@@ -57,6 +57,9 @@ Data-defined route table over the server seam — each route either rewrites the
 | `GET /events?runId=<id>` | Durable SSE stream of normalized events. `Last-Event-ID` header reconnect and `?cursor=` are honored by the server seam; missing `runId` → `400 ERR_PRISM_DEV_ROUTE`. |
 | `GET /runs/:id/replay?cursor=…` | Paged replay of a stored run from the durable `AgentEventSource` — **no session, no provider, no re-execution** (`createPrismAgentEventReplay` page). Returns `{ items, nextCursor?, terminal }`; unknown/foreign run ids → `404`. |
 | `POST /runs/:runId/decisions/:decisionId` | Resumes/denies one suspended approval. Body `{ outcome: "allow_once" \| "allow_always" \| "deny", expectedVersion? }` → forwarded as a single-entry core decision batch; unknown discriminants and stale versions fail closed (`400`) at the core boundary **before any state write**. |
+| `GET /inspect` | Returns the host composition report (`HostCompositionReport`) detailing profile, effective tools, redacted credentials, ownership, storage durability, sandbox capabilities, and governance coverage. | dev composition inspection (`inspectDevInspector`). |
+| `GET /runs/:id/summary` | Projects durable replay events with `projectAgentTimeline({ content: "metadata" })` and returns `summarizeTimeline` (latency, cost, tool counts). No step I/O. Requires `eventSource`. Unknown run → `404`. |
+| `POST /compare` | Body `{ left, right }` each `{ summary, aggregate?, manifest? }` — existing TimelineSummary / ExperimentAggregate fields only. Returns quality/cost/latency winners. `invariantsPassed: false` on either side sets `qualityWinner: "invariant_blocked"`. Bounded 64KiB. |
 
 Reconnect semantics: every SSE frame carries `id: <cursor>`; a reconnecting client sends `Last-Event-ID: <cursor>` and receives exactly the post-cursor events — no duplicates, no loss (server conformance-tested). Replay pages are bounded by the deployment limits (`maxReplayEvents`, `maxReplayCursorBytes`) and ownership-scoped by the source seam itself.
 
@@ -71,6 +74,7 @@ Panels:
 - **Usage** — per-run totals summed from `provider_turn_finished.usage` and the terminal `agent_finished.usage` (input/output/total tokens, cost when the model reports it).
 - **Decisions** — `agent_suspended` renders one card per pending decision (`PendingDecision.approvalId`, tool name, redacted reason, `expectedVersion` from the event's run version). Buttons post `POST /runs/:runId/decisions/:approvalId` ({ outcome: `allow_once` | `allow_always` | `deny`, expectedVersion }); rejections show the seam's fail-closed error verbatim, and a remaining-multi-decision suspension re-renders from the response's `runState.interruption`.
 - **Run selector** — session runs (live + loaded) with status; a durable view of any past run loads via `GET {basePath}/events?runId=…` over `EventSource` — the seam's own `Last-Event-ID` reconnect applies. Without a durable event source wired, loading by runId surfaces that fact instead of pretending to replay.
+- **Compare** — `Compare last 2` loads `/runs/:id/summary` for the two most recent runs and `POST /compare`. Renders quality/cost/latency winners from those artifacts. A high mean score cannot beat a failed invariant. Without durable events the panel says so instead of inventing numbers.
 
 ## Request/response example
 

package/docs/device-adapters.md

@@ -91,6 +91,7 @@ if (chunk.accepted) emit(redactDeviceTelemetry(createSecretRedactor([token]), fr
 ## Related APIs
 
 - [Browser automation](browser-automation.md): verified-state checkpoints + reload/verify-before-side-effect for browser composition.
+- [Realtime voice](realtime-voice.md): governed OpenAI Realtime orchestration on this admission contract.
 - [Linux desktop control](computer-use-linux.md): first-party host-owned `computer-use-linux` MCP wrapper using this contract.
 - [Conversations](conversations.md): durable threads that own the runs device sessions bind to.
 - [Host security](host-security.md): approval, sandbox, and egress trust boundaries device adapters compose over.

package/docs/document-reader.md

@@ -1,8 +1,10 @@
 # Document reader (`@arnilo/prism-coding-tools/document-reader`)
 
+> **Optional peer install:** `pdf-parse` and/or `mammoth` — see [Optional peer dependencies](peer-dependencies.md). OCR uses **native fetch**, not an SDK peer.
+
 ## What it does
 
-Optional bounded literal-text extraction for PDF and DOCX files, consumed by the coding `read` tool (plan 018 closeout `doc-reader`, 0.1.6). `createDocumentReader()` returns a `DocumentReader` that the host wires into `createReadTool(cwd, { documentReader })`; the read tool then extracts text from supported documents instead of falling back to the raw text page.
+Optional bounded literal-text extraction for PDF and DOCX files, consumed by the coding `read` tool (plan 018 closeout `doc-reader`, 0.1.6). `createDocumentReader()` returns a `DocumentReader` that the host wires into `createReadTool(cwd, { documentReader })`; the read tool then extracts text from supported documents instead of falling back to the raw text page. Scanned PDFs/images need a **host-selected** `createMistralOcrParser({ apiKey })` passed in `parsers` — default wiring never calls an external OCR service.
 
 ## When to use it
 
@@ -61,18 +63,26 @@ const myPdfParser: DocumentParser = {
   },
 };
 const reader = await createDocumentReader({ parsers: [myPdfParser, await createPdfParser()] });
+
+import { createMistralOcrParser } from "@arnilo/prism-coding-tools/document-reader";
+const ocr = createMistralOcrParser({
+  apiKey: hostKey, // never read from process.env
+  recordUsage: (u) => router.recordUsage({ /* Task 7 */ tokens: 0, costUsd: hostPrice(u) }),
+});
+const scanned = await createDocumentReader({ parsers: [ocr] }); // not in the default parser list
 ```
 
 ## Extension and configuration notes
 
 - Default parser wiring uses the optional peer dependencies `pdf-parse` (PDF) and `mammoth` (DOCX raw text). Both are declared optional (`peerDependenciesMeta`); `createDocumentReader` fails closed with a documented error at creation when a selected format's peer is absent — never at read time. Hosts pin parser versions (their CVE surface is the host's responsibility; parser advisory is reviewed at ship time).
+- `createMistralOcrParser` is **not** a default parser. It POSTs `https://api.mistral.ai/v1/ocr` (`mistral-ocr-latest`) with inline `data:` URLs (`include_image_base64: false`). No Files API upload, so no remote cleanup. Host `documentUrl` values pass `assertSsrfAllowedUrl`. Extracted markdown is untrusted. Caps: 8 MiB / 32 pages / 60 s / 1 in-flight by default (hard 50 MiB / 10 000 pages / 180 s / 4). Pass `recordUsage` to admit cost through Task 7 accounting. `baseUrl` selects residency.
 - DOCX has no page concept in raw text: `pages` is always `1` and the page cap applies to PDF only; the text cap governs DOCX output.
 - The read tool re-checks `maxTextBytes` on results (parity with its text-page bounds check) and refuses reader output beyond it.
 - The adapter truncates over-cap text at a UTF-8 byte boundary (never splits a code point).
 
 ## Security and performance notes
 
-- No embedded-script execution, no macro evaluation, no external resource fetching — the peer raw-text surfaces are pure extractors, and the no-fetch property is enforced by an egress tripwire test.
+- No embedded-script execution, no macro evaluation, no external resource fetching on the **default** parsers — the peer raw-text surfaces are pure extractors, and the no-fetch property is enforced by an egress tripwire test on `document-reader/index.js`. OCR is a separate module and only runs when the host passes that parser.
 - Decompression/size-bomb protection: the read tool stats and refuses files above `maxBytes` before loading; output is capped at `maxTextBytes`.
 - Extraction envelope (recorded in `scripts/budgets.json` `docReader`, measured 2026-08-11): a max-cap 1000-page PDF (288 KB) extracts in ~162 ms with ~17 MB heap delta; the gate asserts completion within the ceiling or documented refusal.
 - Parser code never receives a buffer whose format gate failed; random binaries never reach a parser.

package/docs/documents.md

@@ -7,7 +7,7 @@ The `@arnilo/prism-office/documents` package provides specification-compliant, A
 It operates on a canonical, typed abstract syntax tree (AST) called the **Prism Document Model** (`DocModel`, `SheetModel`, `DeckModel`):
 - **Pure in-memory doctrine**: Functions accept `Uint8Array` container buffers or typed model objects and emit `Uint8Array` buffers or JSON models. Zero filesystem reads, zero network I/O, zero `process.env` lookups, and zero child process spawns.
 - **Draft-07 JSON Schema validation & slicing**: Full runtime structural validation with transitive closure slicing (`getDocumentModelSchema`) allowing LLM tools and agent prompts to extract minimal, self-contained sub-schemas (e.g. `doc.paragraph`, `doc.table`).
-- **Bidirectional round-trip fidelity**: Prism-generated documents parse back into structurally equivalent models verified against a per-kind equality specification.
+- **Bidirectional round-trip fidelity**: Prism-generated documents parse back into structurally equivalent models verified against a per-kind equality specification. `importDocument` also reports ZIP parts the model drops (macros, comments, media, charts, OLE, pivots) instead of silently omitting them.
 - **Typed model patch engine**: Immutably applies `set`, `insert`, `remove`, and `move` operations to document blocks, worksheet cells, and presentation slides with schema re-validation and an interactive `createPatchHistory` undo/redo stack.
 - **Framework-neutral preview blocks & bounded HTML**: Generates structured snapshots (`PreviewBlock[]`) for native desktop/web UI grids and outline trees, as well as safe, sanitize-by-construction HTML fragments (`renderPreviewHtml`) guaranteed to contain no executable scripts, no active pseudo-protocols, and no external hyperlinks.
 - **Boundary text redaction**: Pluggable `SecretRedactor` hook to sanitize extracted text content (paragraphs, cells, notes, tables) at the parse boundary before models are returned.
@@ -31,11 +31,13 @@ Do **not** use this package for collaborative real-time editing (OT/CRDT), macro
 | --- | --- | --- |
 | `generateDocument` | `(model: DocumentModel, options: GenerateDocumentOptions) => Promise<GenerateDocumentResult>` | Translates a typed model into spec-compliant OOXML binary bytes (PK zip container) with a SHA-256 content hash. |
 | `parseDocument` | `(bytes: Uint8Array, options: ParseDocumentOptions) => Promise<DocumentModel>` | Verifies PK zip signature, enforces caps, translates OOXML parts, applies optional redaction, and returns a validated model. |
+| `importDocument` | `(bytes: Uint8Array, options: ParseDocumentOptions) => Promise<ImportDocumentResult>` | Same parse plus a ZIP-name **fidelity** report of structures the Document Model drops (macros, comments, media, charts, OLE, pivots). `parseDocument` returns `.model` only. |
 | `patchDocument` | `(model: DocumentModel, patches: readonly DocumentPatch[], options?: PatchDocumentOptions) => DocumentModel` | Clones the model, applies typed structural patch operations, and validates the resulting model against Draft-07 schemas. |
+| `diffDocument` | `(from: DocumentModel, to: DocumentModel, options?: DiffDocumentOptions) => DocumentDiff` | Structural paragraph/table/cell/slide diff. Decimal cells compare as canonical strings. Caps report `truncated` instead of unbounded walk. |
 | `createPatchHistory` | `(initialModel: DocumentModel) => PatchHistory` | Creates an interactive undo/redo history manager for host editing workflows. |
 | `renderPreviewBlocks` | `(model: DocumentModel, options?: PreviewBlocksOptions) => PreviewBlock[]` | Emits framework-neutral structured blocks (document outlines, bounded sheet grid chunks, slide summaries). |
 | `renderPreviewHtml` | `(model: DocumentModel, options?: PreviewHtmlOptions) => string` | Emits safe, bounded HTML fragments with all entities escaped and external URLs neutralized. |
-| `getDocumentModelSchema`| `(options: GetDocumentModelSchemaOptions) => Record<string, unknown>` | Retrieves full Draft-07 JSON Schema or a self-contained sliced sub-schema with resolved `$defs`. |
+| `documentModelSchema` | `(kind: DocumentKind, slice?: string \| readonly string[]) => JsonSchema` | Retrieves the Draft-07 JSON Schema for a document kind, or a self-contained sliced sub-schema with resolved `$defs` (`docModelSchema` / `sheetModelSchema` / `deckModelSchema` expose the unsliced schemas). |
 | `validateDocumentModel`| `(model: unknown) => asserts model is DocumentModel` | Validates arbitrary JSON objects against Draft-07 document schemas and structural invariants. |
 
 ### Capacity Limits and Defaults
@@ -139,6 +141,8 @@ console.log(`Generated DOCX (${bytes.byteLength} bytes, SHA-256: ${contentHash})
 
 // 3. Parse OOXML bytes back to a validated model
 const parsed = await parseDocument(bytes, { kind: "doc" });
+const { model, fidelity } = await importDocument(bytes, { kind: "doc" });
+// fidelity.issues[].code: macros | comments | media | …  lost: dropped | approximated
 
 // 4. Apply typed model patches
 const patched = patchDocument(parsed, [
@@ -152,6 +156,9 @@ history.apply([{ op: "set", target: { title: true }, value: "Updated Review" }])
 console.log(history.canUndo()); // true
 const restored = history.undo(); // restored to "Executive Summary" state
 
+import { diffDocument } from "@arnilo/prism-office/documents";
+const diff = diffDocument(parsed, restored, { maxOps: 4096 });
+
 // 6. Generate structured preview blocks & safe HTML
 const blocks = renderPreviewBlocks(restored);
 const htmlSnippet = renderPreviewHtml(restored, { maxHtmlBytes: 256 * 1024 });
@@ -208,7 +215,8 @@ Financial worksheets often require exact decimal representations that JavaScript
 
 ## Related APIs
 
-- [`@arnilo/prism-coding-tools/document-reader`](./document-reader.md): Bounded literal text extraction from PDF and DOCX documents for coding agent tools.
+- [`@arnilo/prism-coding-tools/document-reader`](./document-reader.md): Bounded literal text extraction from PDF and DOCX documents for coding agent tools; optional host-selected Mistral OCR parser.
 - [`@arnilo/prism-core/integrations/work`](./work-tools.md): Microsoft 365 and Google Workspace identity-scoped connectors.
 - [`@arnilo/prism-coding-tools/agent`](./coding-agent-tools.md): Coding tools and file operations.
 - [`@arnilo/prism-core/governance/observability`](./observability.md): OpenTelemetry instrumentation and trace adapters.
+- [Work artifacts and review](work-artifacts-and-review.md): evidence-bound artifact citations and `evidenceDigest` approvals.

package/docs/enterprise-postgres-state.md

@@ -14,7 +14,7 @@
 | ERP messaging | `erpMessaging` | Transactional outbox/inbox markers plus bounded, tenant-scoped at-least-once dispatch (migration 004). |
 | Multi-party approvals | `createPostgresApprovalStore({ pool, schema, authority })` | Immutable approval requests, role/quorum decisions, revocation, bounded delegation, and atomic grant consumption (migration 005). |
 
-`createPostgresEnterpriseState()` opens a host-supplied or adapter-owned `pg` pool, verifies/applies checksum-protected enterprise migrations (`001_enterprise_state`, `002_tool_effects`, `003_router_reservations`, `004_erp_messaging`, `005_erp_approvals`), and returns those stores plus explicit cleanup and close operations. Importing it performs no I/O. It is separate from session/run persistence in [`@arnilo/prism-core/sessions/postgres`](postgres-persistence.md).
+`createPostgresEnterpriseState()` opens a host-supplied or adapter-owned `pg` pool, verifies/applies checksum-protected enterprise migrations (`001_enterprise_state`, `002_tool_effects`, `003_router_reservations`, `004_erp_messaging`, `005_erp_approvals`, `006_aggregate_budgets`), and returns those stores plus explicit cleanup and close operations. Importing it performs no I/O. It is separate from session/run persistence in [`@arnilo/prism-core/sessions/postgres`](postgres-persistence.md).
 
 ## When to use it
 
@@ -90,7 +90,7 @@ Model-router state is asynchronous and owner/principal/provider/model scoped. Su
 }
 ```
 
-A migration creates `prism_policy_decisions`, `prism_evaluations`, `prism_work_idempotency`, three `prism_model_router_*` tables, `prism_erp_outbox`, `prism_erp_inbox`, `prism_erp_approvals`, and its separate `prism_enterprise_migrations` history. Migration `003_router_reservations` adds the nullable-by-default `reservations` JSONB column to `prism_model_router_budgets` (atomic reservation slots for router admission; 0.2.1 readers ignore it). Migration `004_erp_messaging` adds tenant/message and tenant/consumer/message primary keys plus claim, lease, and inbox indexes. Migration `005_erp_approvals` adds the one-row-per-request approval table (PK `tenant_id + id`, status check, decisions JSONB, status/created indexes). Startup serializes per-schema setup with an advisory transaction lock and rejects checksum or catalog drift rather than silently repairing it.
+A migration creates `prism_policy_decisions`, `prism_evaluations`, `prism_work_idempotency`, three `prism_model_router_*` tables, `prism_erp_outbox`, `prism_erp_inbox`, `prism_erp_approvals`, and its separate `prism_enterprise_migrations` history. Migration `003_router_reservations` adds the nullable-by-default `reservations` JSONB column to `prism_model_router_budgets` (atomic reservation slots for router admission; 0.2.1 readers ignore it). Migration `004_erp_messaging` adds tenant/message and tenant/consumer/message primary keys plus claim, lease, and inbox indexes. Migration `005_erp_approvals` adds the one-row-per-request approval table (PK `tenant_id + id`, status check, decisions JSONB, status/created indexes). Migration `006_aggregate_budgets` adds `task_id TEXT NOT NULL DEFAULT ''` and `attributions JSONB NOT NULL DEFAULT '{}'::jsonb` to `prism_model_router_budgets` and creates partial index `prism_model_router_budgets_task_idx` on `(tenant_id, account_key, user_key, principal_id, task_id, window_ms) WHERE task_id <> ''`. Task-scoped budgets route through `provider = ':task:'` and `model = taskId`, grouping retries, fallbacks, child workers, compactions, embeddings, and paid tools into a single atomic budget ceiling with granular attributions. Startup serializes per-schema setup with an advisory transaction lock and rejects checksum or catalog drift rather than silently repairing it.
 
 ## Implementation example