@arnilo/prism 0.3.1 → 0.4.0

package/docs/diagrams.md

@@ -0,0 +1,247 @@
+# Diagramming, draw.io embed client, and mxGraph XML validation (`@arnilo/prism-office/diagrams`)
+
+## What it does
+
+The `@arnilo/prism-office/diagrams` package provides an origin-enforced draw.io / diagrams.net iframe embed client, XXE-safe mxGraph XML validation, and byte-stable deterministic XML canonicalization for content hashing and visual artifact workflows in Prism applications and agent runtimes.
+
+### Core Capabilities
+
+- **Origin-Enforced Embed Client**: `createDrawioEmbed({ iframe, origin })` establishes a typed, secure postMessage bridge between host applications and embedded draw.io editor iframes adhering to the diagrams.net `proto=json` embed protocol.
+- **Dual Inbound Verification Boundary**: Inbound `message` events are verified against **both** the configured origin and `iframe.contentWindow` prior to JSON parsing. Foreign origins, rogue windows, and malformed frames are dropped immediately at the boundary.
+- **Strict Outbound Target Enforcement**: Prohibits wildcard `targetOrigin: "*"` on all outbound actions. Every postMessage transmits exclusively to the exact validated origin.
+- **No Public SaaS Default**: Construction requires an explicit, validated origin string (`https://` or `http://`). No default fallback to public SaaS endpoints (`embed.diagrams.net`) exists; self-hosting is the documented deployment.
+- **XXE & Billion-Laughs Defenses**: `validateDrawioXml` rejects DOCTYPE and ENTITY declarations up-front (`UNSAFE_XML_DECLARATION_PATTERN = /<!\s*(?:DOCTYPE|ENTITY)/i`) with `ERR_PRISM_DIAGRAMS_XXE`, and operates with XML parser entity and HTML entity expansion disabled under strict element, attribute, and byte caps.
+- **Deterministic XML Canonicalization**: `canonicalizeDrawioXml` sorts element attributes lexicographically (`a-z`), standardizes double quotes and entity escaping, and normalizes insignificant whitespace while preserving document sequence, producing byte-identical outputs for SHA-256 content hashing.
+- **Visio Format Exclusion (P12 Guard)**: Detects and rejects Microsoft Visio binary (`.vsd`) and OpenXML (`.vsdx`) inputs across all entrypoints with `ERR_PRISM_DIAGRAMS_UNSUPPORTED_FORMAT`.
+- **Dependency-Free Telemetry Seam (P15)**: Pluggable `DiagramsTelemetry` interface emits `diagrams.validate` and `diagrams.canonicalize` spans tracking byte sizes, element/cell counts, and durations without leaking diagram contents or node text.
+
+## When to use it
+
+Use `@arnilo/prism-office/diagrams` when applications, host workspaces, or autonomous agents need to:
+1. Embed an interactive, self-hosted draw.io / diagrams.net editor inside a web or Electron iframe with strictly enforced cross-origin security.
+2. Coordinate diagram editing lifecycles (`init` handshake, `load`, `save`, `autosave`, `merge`, and `export`).
+3. Execute save-with-preview workflows generating SVG (`xmlsvg`) or PNG (`xmlpng`) visual snapshots from the active editor session.
+4. Validate untrusted agent-generated or user-uploaded mxGraph XML models against structural and memory boundaries before persistence.
+5. Compute deterministic content hashes (`sha256(canonicalizeDrawioXml(xml))`) for version control, caching, and change detection.
+
+Do **not** use this package for:
+- Microsoft Visio format conversion (`.vsd` / `.vsdx` files are explicitly excluded and rejected per P12).
+- Server-side headless diagram rendering without an editor instance (use headless browser automation or containerized export services).
+
+## Inputs / request
+
+### Primary Functions
+
+| Function | Signature | Description |
+| --- | --- | --- |
+| `createDrawioEmbed` | `(options: DrawioEmbedOptions) => DrawioEmbed` | Creates an origin-enforced embed client bound to an iframe element or structural frame. |
+| `validateDrawioXml` | `(xml: string \| Uint8Array, options?: DrawioXmlOptions) => DrawioModelSummary` | Validates mxGraph XML well-formedness, caps, and structure, extracting page/cell/edge metrics. |
+| `canonicalizeDrawioXml` | `(xml: string \| Uint8Array, options?: DrawioCanonicalizeOptions) => string` | Produces byte-stable, attribute-sorted, whitespace-normalized XML for content hashing. |
+| `validateDiagramsOrigin` | `(origin: unknown) => string` | Validates that an origin string is a valid `https:` or `http:` URL origin without paths, queries, hashes, or wildcards. |
+| `assertNotVisio` | `(input: string \| Uint8Array) => void` | Asserts that input is not a Microsoft Visio file, throwing `DiagramsFormatError` if Visio signatures are detected. |
+
+### Message & Options Types
+
+```ts
+export interface DrawioEmbedOptions {
+  readonly iframe: DrawioEmbedFrame;
+  readonly origin: string;
+  readonly messageSource?: DrawioMessageSource;
+  readonly onProtocolError?: (error: DiagramsProtocolError) => void;
+  readonly defaultExportTimeoutMs?: number;
+}
+
+export interface DrawioLoadOptions {
+  readonly xml: string;
+  readonly autosave?: boolean;
+  readonly saveAndExit?: boolean;
+  readonly noSaveBtn?: boolean;
+  readonly noExitBtn?: boolean;
+  readonly title?: string;
+}
+
+export interface DrawioExportOptions {
+  readonly format: "xml" | "xmlsvg" | "xmlpng" | "json" | "png" | "svg";
+  readonly scale?: number;
+  readonly border?: number;
+  readonly xml?: string;
+  readonly embedImages?: boolean;
+  readonly timeoutMs?: number;
+}
+
+export interface DrawioXmlCaps {
+  readonly maxBytes?: number;
+  readonly maxElements?: number;
+  readonly maxAttributes?: number;
+}
+```
+
+### Capacity Limits and Defaults
+
+| Cap | Default | Hard Ceiling | Description |
+| --- | --- | --- | --- |
+| `maxBytes` | 32 MiB (`33,554,432`) | 512 MiB (`536,870,912`) | Maximum input XML string or byte length. |
+| `maxElements` | 100,000 | 500,000 | Maximum total XML element count in diagram tree. |
+| `maxAttributes` | 500,000 | 2,000,000 | Maximum total XML attribute count across elements. |
+| `defaultExportTimeoutMs` | 30,000 ms | Unlimited | Timeout waiting for editor export responses. |
+
+## Outputs / response / events
+
+### Error Hierarchy
+
+All error classes extend `DiagramsError` and carry structured `ERR_PRISM_DIAGRAMS_*` error codes:
+
+| Error Class | Code | Cause / Trigger |
+| --- | --- | --- |
+| `DiagramsOriginError` | `ERR_PRISM_DIAGRAMS_ORIGIN_INVALID` | Origin is empty, wildcard (`*`), malformed URL, or contains forbidden path/query/hash components. |
+| `DiagramsProtocolError` | `ERR_PRISM_DIAGRAMS_PROTOCOL` | Inbound message failed JSON parsing, missing event discriminator, or postMessage issued without contentWindow. |
+| `DiagramsXxeError` | `ERR_PRISM_DIAGRAMS_XXE` | XML input contains forbidden DOCTYPE or ENTITY declaration. |
+| `DiagramsCapError` | `ERR_PRISM_DIAGRAMS_XML_CAP` | XML input exceeds byte length, total element count, or total attribute count caps. |
+| `DiagramsXmlMalformedError` | `ERR_PRISM_DIAGRAMS_XML_MALFORMED` | XML input is truncated or violates XML well-formedness rules. |
+| `DiagramsModelInvalidError` | `ERR_PRISM_DIAGRAMS_XML_INVALID_MODEL` | XML root is neither `<mxfile>` nor `<mxGraphModel>`. |
+| `DiagramsFormatError` | `ERR_PRISM_DIAGRAMS_UNSUPPORTED_FORMAT` | Visio format (.vsd/.vsdx) detected in input (P12). |
+| `DiagramsTimeoutError` | `ERR_PRISM_DIAGRAMS_TIMEOUT` | Export operation timed out waiting for editor response. |
+
+### Output Types
+
+```ts
+export interface DrawioModelSummary {
+  readonly pages: number;
+  readonly cells: number;
+  readonly edges: number;
+  readonly width?: number;
+  readonly height?: number;
+  readonly compressed?: boolean;
+}
+
+export interface DrawioExportResult {
+  readonly format: string;
+  readonly data: string;
+  readonly xml?: string;
+  readonly bounds?: DrawioExportBounds;
+}
+```
+
+## Request/response example
+
+### Protocol Envelope (`proto=json`)
+
+Inbound editor-to-host `message` event:
+```json
+{
+  "event": "save",
+  "xml": "<mxfile host=\"drawio.internal\"><diagram id=\"1\">...</diagram></mxfile>",
+  "exit": false
+}
+```
+
+Outbound host-to-editor action postMessage:
+```json
+{
+  "action": "load",
+  "xml": "<mxfile host=\"drawio.internal\"><diagram id=\"1\">...</diagram></mxfile>",
+  "autosave": 1,
+  "title": "System Architecture"
+}
+```
+
+## Implementation example
+
+```ts
+import { createDrawioEmbed, validateDrawioXml, canonicalizeDrawioXml } from "@arnilo/prism-office/diagrams";
+
+// 1. Initialize embed client with strict origin binding
+const embed = createDrawioEmbed({
+  iframe: document.getElementById("drawio-frame") as HTMLIFrameElement,
+  origin: "https://drawio.internal.example",
+  onProtocolError(error) {
+    console.error("Protocol error:", error.message);
+  },
+});
+
+// 2. Register typed event listeners
+embed.on("init", () => {
+  const initialXml = `<mxfile host="drawio.internal"><diagram id="d1" name="Architecture"><mxGraphModel dx="800" dy="600"><root><mxCell id="0"/><mxCell id="1" parent="0"/><mxCell id="2" value="Core Agent" vertex="1" parent="1"><mxGeometry x="100" y="100" width="120" height="60" as="geometry"/></mxCell></root></mxGraphModel></diagram></mxfile>`;
+  embed.load({ xml: initialXml, autosave: true });
+});
+
+embed.on("save", async ({ xml, exit }) => {
+  // Validate model before persisting
+  const summary = validateDrawioXml(xml);
+  console.log(`Validated diagram with ${summary.cells} cells and ${summary.edges} edges`);
+
+  // Compute canonical hash for content-addressed storage
+  const canonical = canonicalizeDrawioXml(xml);
+  console.log("Canonical XML ready for persistence");
+
+  if (exit) {
+    console.log("Editor exit requested by user");
+  }
+});
+
+embed.on("autosave", ({ xml }) => {
+  console.log("Draft autosaved:", xml.length, "bytes");
+});
+
+// 3. Save-with-preview flow: export SVG snapshot
+const preview = await embed.export({ format: "xmlsvg" });
+console.log("Exported preview format:", preview.format, "data:", preview.data.slice(0, 40));
+```
+
+## Extension and configuration notes
+
+### Self-Hosted draw.io Deployment
+
+The recommended and supported deployment is a self-hosted instance of the Apache-2.0 `jgraph/drawio` container:
+
+```bash
+docker run -d -p 8080:8080 -e DRAWIO_SERVER_URL="http://localhost:8080" jgraph/drawio
+```
+
+Iframe embed URLs are formed with query parameters configuring json protocol mode:
+```text
+http://localhost:8080/?embed=1&proto=json&spin=1
+```
+
+### Decoupled Structural DOM Interface
+
+`DrawioEmbedFrame` is typed structurally:
+```ts
+export interface DrawioEmbedFrame {
+  readonly contentWindow: {
+    postMessage(message: unknown, targetOrigin: string): void;
+  } | null;
+}
+```
+This enables use with vanilla DOM elements, React/Svelte/Vue refs, Electron webviews, and headless test doubles without requiring browser globals or `@types/dom`.
+
+### OpenTelemetry Telemetry Seam
+
+Pass an optional `DiagramsTelemetry` implementation to record spans without runtime overhead:
+```ts
+const summary = validateDrawioXml(xml, {
+  telemetry: {
+    startSpan(name, attributes) {
+      // Maps to OTel tracer.startSpan("diagrams.validate", { attributes })
+      // Diagram text and labels are NEVER passed to spans.
+      return activeSpan;
+    },
+  },
+});
+```
+
+## Security and performance notes
+
+- **Trust Boundary Placement**: Origin and `event.source` checks execute inside the embed client before JSON parsing. Rogue cross-origin messages and foreign window messages are dropped without triggering listeners or error handlers.
+- **Wildcard Prohibition**: `targetOrigin: "*"` is blocked at construction and runtime. Outbound messages are posted exclusively to the verified origin.
+- **XXE Prevention**: DOCTYPE and ENTITY declarations are rejected up-front by regex pre-checks; `fast-xml-parser` is configured with entity expansion disabled.
+- **Memory Caps**: Progressive limits on byte size, element counts, and attribute counts prevent XML decompression bombs and heap exhaustion.
+- **Single-Pass Performance**: Canonicalization and validation process 1 MB XML models in under 10 ms.
+
+## Related APIs
+
+- [`@arnilo/prism-office/documents`](./documents.md): Specification-compliant OpenXML document generation and preview rendering for DOCX, XLSX, and PPTX.
+- [`@arnilo/prism-office/sheets`](./sheets.md): Spreadsheet and CSV parsing engine with strict financial decimal safety guarantees.
+- [`@arnilo/prism-web-tools/browser`](./browser-automation.md): Browser automation tools and quarantine lifecycle.
+- [`@arnilo/prism-ag-ui`](./ag-ui.md): Agent-User Interface projection and timeline components.
+- [`@arnilo/prism-observability-opentelemetry`](./observability.md): OpenTelemetry instrumentation and trace adapters.

package/docs/documents.md

@@ -0,0 +1,213 @@
+# Documents, spreadsheets, and presentations (`@arnilo/prism-office/documents`)
+
+## What it does
+
+The `@arnilo/prism-office/documents` package provides specification-compliant, AI-native OpenXML document generation, parsing, patching, and bounded preview rendering for Microsoft Word (`.docx`), Microsoft Excel (`.xlsx`), and Microsoft PowerPoint (`.pptx`) artifacts.
+
+It operates on a canonical, typed abstract syntax tree (AST) called the **Prism Document Model** (`DocModel`, `SheetModel`, `DeckModel`):
+- **Pure in-memory doctrine**: Functions accept `Uint8Array` container buffers or typed model objects and emit `Uint8Array` buffers or JSON models. Zero filesystem reads, zero network I/O, zero `process.env` lookups, and zero child process spawns.
+- **Draft-07 JSON Schema validation & slicing**: Full runtime structural validation with transitive closure slicing (`getDocumentModelSchema`) allowing LLM tools and agent prompts to extract minimal, self-contained sub-schemas (e.g. `doc.paragraph`, `doc.table`).
+- **Bidirectional round-trip fidelity**: Prism-generated documents parse back into structurally equivalent models verified against a per-kind equality specification.
+- **Typed model patch engine**: Immutably applies `set`, `insert`, `remove`, and `move` operations to document blocks, worksheet cells, and presentation slides with schema re-validation and an interactive `createPatchHistory` undo/redo stack.
+- **Framework-neutral preview blocks & bounded HTML**: Generates structured snapshots (`PreviewBlock[]`) for native desktop/web UI grids and outline trees, as well as safe, sanitize-by-construction HTML fragments (`renderPreviewHtml`) guaranteed to contain no executable scripts, no active pseudo-protocols, and no external hyperlinks.
+- **Boundary text redaction**: Pluggable `SecretRedactor` hook to sanitize extracted text content (paragraphs, cells, notes, tables) at the parse boundary before models are returned.
+- **Dependency-free telemetry seam**: Zero-overhead `DocumentsTelemetry` hook for OpenTelemetry-compatible span tracing without leaking document or chunk text into telemetry collectors.
+
+## When to use it
+
+Use `@arnilo/prism-office/documents` whenever autonomous agents, coding assistants, workflow orchestrators, or enterprise applications need to:
+1. Synthesize professional DOCX reports, financial XLSX spreadsheets, or PPTX presentation decks from structured LLM outputs.
+2. Ingest existing OOXML artifacts into a structured, validated document model for analysis or automated summarization.
+3. Perform atomic, validated updates or localized edits to documents using typed patch operations.
+4. Render safe, bounded HTML previews or framework-neutral outline and grid snapshots in web and desktop hosts.
+
+Do **not** use this package for collaborative real-time editing (OT/CRDT), macro execution, or in-memory spreadsheet formula evaluation (formulas are preserved verbatim as `{ formula, cachedValue? }` read-only pairs).
+
+## Inputs / request
+
+### Generation & Parsing Functions
+
+| Function | Signature | Description |
+| --- | --- | --- |
+| `generateDocument` | `(model: DocumentModel, options: GenerateDocumentOptions) => Promise<GenerateDocumentResult>` | Translates a typed model into spec-compliant OOXML binary bytes (PK zip container) with a SHA-256 content hash. |
+| `parseDocument` | `(bytes: Uint8Array, options: ParseDocumentOptions) => Promise<DocumentModel>` | Verifies PK zip signature, enforces caps, translates OOXML parts, applies optional redaction, and returns a validated model. |
+| `patchDocument` | `(model: DocumentModel, patches: readonly DocumentPatch[], options?: PatchDocumentOptions) => DocumentModel` | Clones the model, applies typed structural patch operations, and validates the resulting model against Draft-07 schemas. |
+| `createPatchHistory` | `(initialModel: DocumentModel) => PatchHistory` | Creates an interactive undo/redo history manager for host editing workflows. |
+| `renderPreviewBlocks` | `(model: DocumentModel, options?: PreviewBlocksOptions) => PreviewBlock[]` | Emits framework-neutral structured blocks (document outlines, bounded sheet grid chunks, slide summaries). |
+| `renderPreviewHtml` | `(model: DocumentModel, options?: PreviewHtmlOptions) => string` | Emits safe, bounded HTML fragments with all entities escaped and external URLs neutralized. |
+| `getDocumentModelSchema`| `(options: GetDocumentModelSchemaOptions) => Record<string, unknown>` | Retrieves full Draft-07 JSON Schema or a self-contained sliced sub-schema with resolved `$defs`. |
+| `validateDocumentModel`| `(model: unknown) => asserts model is DocumentModel` | Validates arbitrary JSON objects against Draft-07 document schemas and structural invariants. |
+
+### Capacity Limits and Defaults
+
+Caps are strictly enforced in memory before compute-intensive translation or parsing:
+
+| Cap | Default | Hard Ceiling | Target |
+| --- | --- | --- | --- |
+| `maxBytes` | 32 MiB (`33,554,432`) | 512 MiB (`536,870,912`) | Binary input buffer & generated OOXML bytes |
+| `maxBlocks` | 10,000 | 50,000 | Total blocks in a `DocModel` |
+| `maxSheets` | 50 | 250 | Worksheets in a `SheetModel` |
+| `maxRows` | 100,000 | 1,000,000 | Rows per worksheet |
+| `maxColumns` | 1,000 | 16,384 | Columns per worksheet |
+| `maxCells` | 1,000,000 | 10,000,000 | Total cells across all worksheets |
+| `maxSlides` | 500 | 2,000 | Total slides in a `DeckModel` |
+| `maxParagraphs` | 10,000 | 50,000 | Total paragraphs in a document |
+| `maxRuns` | 50,000 | 250,000 | Total formatted text runs in a document |
+
+## Outputs / response / events
+
+### Error Hierarchy
+
+All package operations throw typed exceptions derived from `DocumentsError`:
+
+| Error Class | Error Code | Trigger Condition |
+| --- | --- | --- |
+| `DocumentsValidationError` | `ERR_PRISM_DOCUMENTS_INVALID_MODEL` | Input JSON fails Draft-07 schema validation, has dimension mismatches, or invalid decimal string syntax. |
+| `DocumentsCapExceededError` | `ERR_PRISM_DOCUMENTS_CAP_EXCEEDED` | Document exceeds byte size, block count, cell count, or slide count caps. |
+| `DocumentsParseError` | `ERR_PRISM_DOCUMENTS_PARSE_FAILED` | Input buffer lacks PK zip signature, is corrupted, or fails OOXML part parsing. |
+| `DocumentsFormatError` | `ERR_PRISM_DOCUMENTS_UNSUPPORTED_FORMAT` | Format mismatch (e.g. attempting to generate PPTX from a `DocModel`). |
+| `DocumentsPatchError` | `ERR_PRISM_DOCUMENTS_PATCH_FAILED` | Out-of-bounds index target, unknown patch operation, or invalid patch structure. |
+
+## Request/response example
+
+### JSON Schema Document Models
+
+```json
+{
+  "kind": "sheet",
+  "modelVersion": 1,
+  "title": "Q3 Financial Summary",
+  "sheets": [
+    {
+      "name": "Revenue",
+      "columnWidths": [{ "column": 0, "width": 25 }, { "column": 1, "width": 15 }],
+      "frozenPanes": { "rows": 1, "columns": 0 },
+      "cells": [
+        ["Department", "Operating Budget", "Actual Expense"],
+        ["Engineering", { "type": "decimal", "value": "1500000.00" }, { "type": "decimal", "value": "1420000.50" }],
+        ["Operations", { "type": "decimal", "value": "450000.00" }, { "type": "decimal", "value": "435000.00" }],
+        ["Total", { "formula": "=SUM(B2:B3)", "cachedValue": 1950000 }, { "formula": "=SUM(C2:C3)", "cachedValue": 1855000.5 }]
+      ]
+    }
+  ]
+}
+```
+
+## Implementation example
+
+```ts
+import {
+  generateDocument,
+  parseDocument,
+  patchDocument,
+  createPatchHistory,
+  renderPreviewBlocks,
+  renderPreviewHtml,
+  type DocModel,
+} from "@arnilo/prism-office/documents";
+
+// 1. Define typed document model
+const doc: DocModel = {
+  kind: "doc",
+  modelVersion: 1,
+  title: "Architecture Review",
+  blocks: [
+    { type: "heading", level: 1, text: "System Architecture" },
+    {
+      type: "paragraph",
+      runs: [
+        { text: "Prism operates purely in memory with ", bold: false },
+        { text: "zero-trust security boundaries.", bold: true, italic: true },
+      ],
+    },
+    {
+      type: "table",
+      rows: 2,
+      columns: 2,
+      cells: [
+        ["Module", "Latency"],
+        ["Parser", "12ms"],
+      ],
+    },
+  ],
+};
+
+// 2. Generate in-memory DOCX binary with SHA-256 contentHash
+const { bytes, contentHash } = await generateDocument(doc, { format: "docx" });
+console.log(`Generated DOCX (${bytes.byteLength} bytes, SHA-256: ${contentHash})`);
+
+// 3. Parse OOXML bytes back to a validated model
+const parsed = await parseDocument(bytes, { kind: "doc" });
+
+// 4. Apply typed model patches
+const patched = patchDocument(parsed, [
+  { op: "set", target: { block: 0 }, block: { type: "heading", level: 1, text: "Executive Summary" } },
+  { op: "insert", target: { afterBlock: 0 }, block: { type: "paragraph", text: "New introduction." } },
+]);
+
+// 5. Interactive Undo/Redo editing
+const history = createPatchHistory(patched);
+history.apply([{ op: "set", target: { title: true }, value: "Updated Review" }]);
+console.log(history.canUndo()); // true
+const restored = history.undo(); // restored to "Executive Summary" state
+
+// 6. Generate structured preview blocks & safe HTML
+const blocks = renderPreviewBlocks(restored);
+const htmlSnippet = renderPreviewHtml(restored, { maxHtmlBytes: 256 * 1024 });
+```
+
+## Extension and configuration notes
+
+### Sub-package Pinning
+To avoid pulling in CLI frameworks or extraneous dependencies, `@arnilo/prism-office/documents` directly pins the exact underlying modular packages:
+- `@office-open/docx@0.12.3`
+- `@office-open/xlsx@0.12.3`
+- `@office-open/pptx@0.12.3`
+
+### Draft-07 JSON Schema Slicing
+For AI agent tool generation where token budgets are constrained, `getDocumentModelSchema` provides closure slicing:
+```ts
+// Returns only TableBlock, CellValue, and their transitively required definitions in $defs
+const tableSchema = getDocumentModelSchema({ kind: "doc", slice: "table" });
+```
+
+### Text Redaction Hook (`SecretRedactor`)
+When parsing untrusted or sensitive OOXML containers, hosts can inject a redaction hook to scrub PII or secrets before the document model enters memory:
+```ts
+const sanitizedModel = await parseDocument(rawBytes, {
+  kind: "doc",
+  redactor: {
+    redact: (text: string) => text.replace(/\b\d{3}-\d{2}-\d{4}\b/g, "[REDACTED-SSN]"),
+  },
+});
+```
+
+### Telemetry Seam
+The dependency-free `DocumentsTelemetry` seam allows optional OpenTelemetry instrumentation without adding runtime telemetry dependencies:
+```ts
+const telemetry: DocumentsTelemetry = {
+  startSpan(name, attributes) {
+    // Maps to tracer.startSpan with allow-listed metadata (kind, format, bytes, counts)
+    // Model content and document text are NEVER passed to telemetry spans.
+    return activeSpan;
+  },
+};
+```
+
+### Decimal Fidelity Ceiling
+Financial worksheets often require exact decimal representations that JavaScript 64-bit binary floating-point numbers cannot represent without precision loss. `@arnilo/prism-office/documents` supports canonical string decimals (`{ type: "decimal", value: "1500000.00" }`), preserving exact numerical formatting without IEEE 754 precision artifacts.
+
+## Security and performance notes
+
+- **Pure In-Memory Operation**: No temporary files, no shell execution, no binary spawning, and zero network sockets.
+- **ZIP Signature Gating**: Buffers must begin with PK zip container signatures (`0x50, 0x4B, 0x03, 0x04`). Extension-based type inference is strictly prohibited.
+- **Fail-Closed Caps**: Input size and element count caps are evaluated before entering XML translation passes, preventing zip-bomb and decompression amplification attacks.
+- **Sanitize-by-Construction HTML**: `renderPreviewHtml` strictly entity-encodes all text fields, neutralizes dangerous protocols (`javascript:`, `http://`, `https://`), strips raw script/image tags, and caps output size to prevent DOM-based XSS and memory exhaustion.
+- **Performance Budget**: Warm generation of 200-block documents completes in under 15 ms; parse and round-trip equality checks complete in under 100 ms.
+
+## Related APIs
+
+- [`@arnilo/prism-document-reader`](./document-reader.md): Bounded literal text extraction from PDF and DOCX documents for coding agent tools.
+- [`@arnilo/prism-work-tools`](./work-tools.md): Microsoft 365 and Google Workspace identity-scoped connectors.
+- [`@arnilo/prism-coding-agent`](./coding-agent-tools.md): Coding tools and file operations.
+- [`@arnilo/prism-observability-opentelemetry`](./observability.md): OpenTelemetry instrumentation and trace adapters.

package/docs/evaluations.md

@@ -19,6 +19,7 @@ Use this package when a host needs offline quality checks or sampled live scorin
 | `createMemoryEvaluationStore` | optional seed records |
 | `appendEvaluationFeedback` | `RunFeedbackStore`, `EvaluationStore`, feedback fields, and 1–64 known evaluation IDs |
 | `createPersistenceTraceResolver` | explicit `ProductionPersistenceStore`, exact session/run/ownership, page/byte bounds |
+| `datasetFromRuns` | `runIds` and/or `sessionIds`, existing dataset, `ProductionPersistenceStore`, ownership, redactor/secrets, optional `toItem` |
 | `createModelJudge` | host judge callback, stable rubric/version, timeout/attempt/output bounds |
 | `runComparison` | immutable dataset, 2–8 named candidates by default, pairwise scorers |
 | `assertEvaluationThreshold` / `serializeEvaluationReport` | mean/failure/per-scorer gates and bounded redacted JSON |
@@ -32,6 +33,7 @@ Use this package when a host needs offline quality checks or sampled live scorin
 | `runExperiment` | `ExperimentReport` with stable item order, evaluations, and aggregates |
 | `EvaluationStore.query` | cursor-paginated, ownership-filtered page |
 | `appendEvaluationFeedback` | immutable `RunFeedbackRecord` containing only evaluation/scorer IDs |
+| `datasetFromRuns` | `{ dataset, version, added, skipped }` — new immutable dataset version with one item per added run; skips carry reasons (missing run, ownership mismatch, empty output) |
 
 ## Request/response example
 
@@ -121,6 +123,7 @@ console.log(report.aggregate.meanScore, linked.evaluationIds);
 - Model judges are host callbacks, not providers: Prism passes rubric/version plus bounded target only—never credential resolvers, tools, or workspace. Defaults are one attempt, 30 seconds, and 16 KiB output; failures become redacted evaluation records.
 - Pairwise candidates are sorted by name, executed once per item, compared in stable item/pair/scorer order, and record ties/failures without choosing a winner. Candidate and scorer outputs have byte caps.
 - `assertEvaluationThreshold()` throws `ERR_PRISM_EVAL_THRESHOLD`; an uncaught error gives CI a non-zero exit. Keep model-judge/live gates credential-gated and outside the network-free default suite. `serializeEvaluationReport()` bounds/redacts checked-in artifacts.
+- `datasetFromRuns` redacts before item construction and never stores an unredactable item; curated items are byte-capped (`ERR_PRISM_EVAL_CURATE`), ownership is verified exactly per run, and a failed append leaves the prior dataset version untouched.
 
 ## Trace, judge, comparison, and CI example
 
@@ -137,12 +140,43 @@ assertEvaluationThreshold(report, { minimumMean: 0.9, maximumFailures: 0 });
 
 `traceResolver` is explicit; no arbitrary run search occurs. `baseline`/`candidate` are host functions returning `AgentRunResult`. See `examples/evaluation-gate.ts` for a network-free gate and `examples/coding-browser-evaluation.ts` for coding/browser adversarial fixtures.
 
+## Curating datasets from production runs
+
+Plan 043 adds `datasetFromRuns`: it turns recorded runs (production incident transcripts included) into dataset items in one call — resolve through `createPersistenceTraceResolver`, redact, map through an optional host `toItem`, and append as a **new immutable dataset version**. The prior version object is never mutated.
+
+```ts
+import { datasetFromRuns, defineDataset } from "@arnilo/prism-evals";
+
+const result = await datasetFromRuns({
+  runIds: ["run_9f2", "run_a71"],
+  dataset: defineDataset({ id: "support-regressions", items: [] }),
+  store: persistence,
+  ownership: { tenantId: "t1", userId: "u1" },
+  redactor,
+  toItem: (run) => ({
+    input: run.input,
+    expected: run.feedback?.metadata?.expected, // human-graded gold from the feedback seam
+    metadata: { graded: run.feedback?.rating },
+  }),
+});
+// result.added === 2, result.dataset.version === "2"
+```
+
+Omit `toItem` and the default mapping is used: `input` = first user message, `expected` = the recorded feedback's `metadata.expected` when present, `output` = final assistant text under `metadata.output`.
+
+- Session ids (`sessionIds`) expand to every run recorded under the session; bare `runIds` are located by one ownership-bounded scan of the run ledger (page/byte caps apply).
+- Each resolved run maps to one item keyed by the run id. The default `toItem` carries the first user message as `input`; `expected` comes **only from the feedback seam** (`metadata.expected` of the latest human-graded `RunFeedbackRecord`) — never re-derived from untrusted outputs, and omitted entirely (not fabricated) when a run has no feedback. The recorded output rides `metadata.output` for provenance. A host `toItem` always wins and may return `undefined` to drop a run (`host filter` skip).
+- Feedback costs one bounded, owner-scoped query per curation batch (`store.feedback`); records are read id-only per the feedback linkage contract — scorer payloads are never copied. A feedback query failure (e.g. scope contract mismatch) omits `expected` instead of aborting the batch.
+- Every item field passes the host `redactor` after host mapping — fail closed: a redactor failure or an item over the frozen 4 MiB cap (`ERR_PRISM_EVAL_CURATE`) skips the run or aborts the append before anything is persisted. Cross-tenant runs are never readable (resolver ownership check → `ownership mismatch` skip).
+
+Prompt versions can ride the same primitives: [`assertPromptPromotion`](prompt-registry.md#eval-gated-promotion) in `@arnilo/prism-prompts` resolves two prompt versions, runs them through `runComparison`, and returns a `promote`/`hold` verdict with per-scorer aggregates and a bounded report — never applying the change itself.
+
 ## Coding and browser adversarial evaluations (0.0.9)
 
 Release 0.0.9 ships curated network-free adversarial fixtures in package tests:
 
 - `@arnilo/prism-coding-agent` `eval-fixtures.test.ts`: safe native list vs shell, Git path/ref injection, dirty-tree rollback, unknown named-check failure, PR-handoff artifact completeness, and prompt-injection file content under read-only tools.
-- `@arnilo/prism-browser` `eval-fixtures.test.ts`: stale snapshot refs, side-effect approval, private/loopback/file deny, upload/download/screenshot policy, CSS/evaluate target rejection, and hostile accessible-name text.
+- `browser` `eval-fixtures.test.ts`: stale snapshot refs, side-effect approval, private/loopback/file deny, upload/download/screenshot policy, CSS/evaluate target rejection, and hostile accessible-name text.
 
 Fixtures reuse `@arnilo/prism-evals` (`defineDataset` / `defineScorer` / `scoreRun` / `assertEvaluationThreshold` / `serializeEvaluationReport`). Optional SWE-bench-compatible or live-browser harnesses remain host adapters — they are not default dependencies or quality claims. Protected real Docker/Playwright gates stay env-gated (`PRISM_TEST_DOCKER_SANDBOX`, `PRISM_LIVE_PLAYWRIGHT`) and never enter `sdk:ready`.
 

package/docs/extension-authoring.md

@@ -168,6 +168,48 @@ await agent.createSession().run("Use the Acme extension.", { activeSkills: ["acm
 - Middleware from `api.use()` runs only when the host passes `kernel.middleware` into runtime configuration.
 - Provider packages, provider request policies, system prompt contributions, instruction injectors, builders, strategies, commands, store factories, resource loaders, settings providers, and credential resolvers are all inert until host code selects or invokes them.
 
+### Host driver hooks (opt-in)
+
+A contributed command can act — start a session run, start a workflow, steer
+an active run — only when the **host** injects driver capabilities into the
+execution context. Drivers are never package-supplied: a command that wants
+them guards on `context.drivers` and degrades gracefully when the host
+supplies none. Commands stay inert data in hosts without drivers, and the
+context shape is unchanged (no `drivers` key at all).
+
+```ts
+// Host opt-in (e.g. RPC session factory):
+await runRpcServer({
+  stdin,
+  stdout,
+  createSession,
+  commands,
+  drivers: {
+    startRun: (input, options) => session.run(input, options),
+    startWorkflow: (workflow, input, options) => runWorkflow(workflow, input, options),
+    steer: (runId, input) => session.steer(runId, input),
+  },
+});
+
+// Contributed command (host-opt-in capability use):
+registerCommand({
+  name: "acme.start",
+  async execute(args, context) {
+    if (!context.drivers?.startWorkflow) {
+      return { name: "acme.start", error: { message: "host did not supply workflow drivers" } };
+    }
+    const run = await context.drivers.startWorkflow(workflowFor(args), args.input);
+    return { name: "acme.start", value: { runId: run.runId, status: run.status } };
+  },
+});
+```
+
+`CommandDrivers` is typed (`startRun` / `startWorkflow` / `steer`) and exported
+from the core contracts surface. Driver errors surface through the command's
+normal error path — commands map failures to `CommandResult.error`
+(`ErrorInfo`) or let the host error envelope carry them. Driver presence does
+not affect command `metadata.trust` labeling.
+
 ## Security and performance notes
 
 - Prism does not sandbox extension code. Hosts should load only trusted packages or run untrusted packages in their own sandbox/process before calling Prism APIs.

package/docs/graft.md

@@ -2,7 +2,7 @@
 
 ## What it does
 
-`@arnilo/prism-graft` is an optional package that wires [nanonets/graft](https://github.com/nanonets/graft) — a repository context-graph CLI (`graft/` directory, INDEX.md orientation, symbol-level wiring graph) — into Prism contribution contracts.
+`@arnilo/prism-memory/graft` is an optional subpath that wires [nanonets/graft](https://github.com/nanonets/graft) — a repository context-graph CLI (`graft/` directory, INDEX.md orientation, symbol-level wiring graph) — into Prism contribution contracts.
 
 It registers six pull tools backed by the graft CLI (`--json`, argv-safe), a push-mode retrieval-pack context provider plus first-turn orientation injector carried on the `graft` skill, commands (`graft`, `graft-build`, `graft-check`, `graft-viz`), and an edit-watch middleware that computes blast radius after mutating tool calls. Import is inert; a missing graft CLI fails closed at `setup` with a bounded redacted error.
 
@@ -74,7 +74,7 @@ Tool call (pull):
 Status event:
 
 ```json
-{ "type": "graft:status", "extension": "@arnilo/prism-graft", "metadata": { "fresh": true, "missing": 0, "stale": 2 } }
+{ "type": "graft:status", "extension": "@arnilo/prism-memory/graft", "metadata": { "fresh": true, "missing": 0, "stale": 2 } }
 ```
 
 ## Implementation example
@@ -83,7 +83,7 @@ See [`examples/graft-extension.ts`](../examples/graft-extension.ts) — network-
 
 ```ts
 import { createExtensionKernel, createMemorySessionStore } from "@arnilo/prism";
-import { createGraftExtension } from "@arnilo/prism-graft";
+import { createGraftExtension } from "@arnilo/prism-memory/graft";
 
 const store = createMemorySessionStore();
 const kernel = createExtensionKernel({ errorPolicy: "throw" });

package/docs/guardrails.md

@@ -78,7 +78,7 @@ Guardrails are callbacks supplied by the host. Prism does not discover, load, re
 
 Optional `@arnilo/prism-policy` can record guardrail outcomes via `recordGuardrailDecision` (evidence refs only; see [Policy and audit](policy-and-audit.md)).
 
-Output buffering prevents blocked provider content from reaching subscribers, session entries, ledgers, parsers, delegation, or tools. Tool-output checks receive raw results but Prism discards blocked raw output before event, ledger, transcript, or MCP exposure. Redaction replaces exact known values only; it is not general secret detection. Parallel checks receive an abort signal, but callback code must honor it to stop in-flight work. Browser snapshots and page text from `@arnilo/prism-browser` are untrusted external content: never allow them to modify tools, permissions, credentials, or policy. Browser mutations still require host `ExecutionPolicy`/approval; prompt-injection text in a page cannot grant upload/download release.
+Output buffering prevents blocked provider content from reaching subscribers, session entries, ledgers, parsers, delegation, or tools. Tool-output checks receive raw results but Prism discards blocked raw output before event, ledger, transcript, or MCP exposure. Redaction replaces exact known values only; it is not general secret detection. Parallel checks receive an abort signal, but callback code must honor it to stop in-flight work. Browser snapshots and page text from the `browser` subpath are untrusted external content: never allow them to modify tools, permissions, credentials, or policy. Browser mutations still require host `ExecutionPolicy`/approval; prompt-injection text in a page cannot grant upload/download release.
 
 ## Related APIs