@arnilo/prism 0.3.2 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (116) hide show
  1. package/CHANGELOG.md +17 -0
  2. package/README.md +34 -57
  3. package/dist/agent-run-lifecycle.js +4 -0
  4. package/dist/agent-run-state.d.ts +4 -0
  5. package/dist/agent-run-state.js +18 -5
  6. package/dist/agent-session/session.d.ts +7 -0
  7. package/dist/agent-session/session.js +59 -2
  8. package/dist/cli-dev.d.ts +29 -0
  9. package/dist/cli-dev.js +52 -0
  10. package/dist/cli-init.d.ts +17 -2
  11. package/dist/cli-init.js +194 -21
  12. package/dist/cli-runner.d.ts +5 -1
  13. package/dist/cli-runner.js +12 -1
  14. package/dist/contracts-core/agent.d.ts +6 -0
  15. package/dist/contracts-protocol.d.ts +18 -0
  16. package/dist/contracts-run-state.d.ts +1 -2
  17. package/dist/index.d.ts +3 -1
  18. package/dist/index.js +2 -1
  19. package/dist/input.d.ts +8 -0
  20. package/dist/input.js +4 -0
  21. package/dist/testing/persistence-schema.d.ts +1 -1
  22. package/dist/testing/persistence-schema.js +32 -28
  23. package/dist/testing/tool-conformance.d.ts +25 -0
  24. package/dist/testing/tool-conformance.js +128 -1
  25. package/dist/tool-search.d.ts +76 -0
  26. package/dist/tool-search.js +199 -0
  27. package/docs/0.1.0-readiness.md +2 -2
  28. package/docs/acp-agent.md +1 -1
  29. package/docs/antigravity-agent.md +1 -1
  30. package/docs/browser-automation.md +5 -5
  31. package/docs/caveman.md +2 -2
  32. package/docs/cli-rpc.md +26 -3
  33. package/docs/coding-security.md +1 -1
  34. package/docs/coding-tools.md +82 -0
  35. package/docs/compaction-and-retry.md +2 -2
  36. package/docs/compaction-llm.md +4 -4
  37. package/docs/compaction-observational-memory.md +3 -3
  38. package/docs/context-and-skills.md +2 -0
  39. package/docs/core.md +85 -0
  40. package/docs/credential-storage.md +1 -1
  41. package/docs/database-persistence.md +4 -0
  42. package/docs/dev-inspector.md +103 -0
  43. package/docs/diagrams.md +247 -0
  44. package/docs/documents.md +213 -0
  45. package/docs/evaluations.md +35 -1
  46. package/docs/graft.md +3 -3
  47. package/docs/guardrails.md +1 -1
  48. package/docs/host-security.md +4 -3
  49. package/docs/impeccable.md +2 -2
  50. package/docs/index.md +31 -20
  51. package/docs/mcp-tools.md +1 -1
  52. package/docs/migrate-to-0.4.md +312 -0
  53. package/docs/migration.md +22 -0
  54. package/docs/model-routing.md +1 -1
  55. package/docs/multi-agent-patterns.md +177 -0
  56. package/docs/multimodal-content.md +1 -1
  57. package/docs/obscura.md +10 -10
  58. package/docs/openapi-tools.md +1 -1
  59. package/docs/performance.md +23 -3
  60. package/docs/persistence-credentials-multimodality-primitives.md +1 -1
  61. package/docs/policy-and-audit.md +1 -1
  62. package/docs/ponytail.md +2 -2
  63. package/docs/prompt-registry.md +106 -0
  64. package/docs/provider-caching.md +32 -32
  65. package/docs/provider-conformance.md +1 -1
  66. package/docs/provider-packages.md +19 -19
  67. package/docs/provider-primitives.md +4 -4
  68. package/docs/providers/ai-sdk.md +3 -3
  69. package/docs/providers/alibaba.md +5 -5
  70. package/docs/providers/anthropic.md +6 -6
  71. package/docs/providers/azure.md +3 -3
  72. package/docs/providers/bedrock.md +3 -3
  73. package/docs/providers/clinepass.md +3 -3
  74. package/docs/providers/deepseek.md +3 -3
  75. package/docs/providers/google.md +4 -4
  76. package/docs/providers/kimi.md +3 -3
  77. package/docs/providers/neuralwatt.md +8 -8
  78. package/docs/providers/ollama.md +3 -3
  79. package/docs/providers/openai-compatible.md +1 -1
  80. package/docs/providers/openai.md +5 -5
  81. package/docs/providers/opencode-go.md +4 -4
  82. package/docs/providers/openrouter.md +3 -3
  83. package/docs/providers/vertex.md +5 -5
  84. package/docs/providers/xai.md +3 -3
  85. package/docs/providers/zai.md +3 -3
  86. package/docs/rag.md +5 -5
  87. package/docs/release-and-install.md +98 -50
  88. package/docs/runs-and-usage.md +14 -1
  89. package/docs/server.md +90 -1
  90. package/docs/sheets.md +229 -0
  91. package/docs/supervisors.md +1 -0
  92. package/docs/thinking-and-reasoning.md +10 -10
  93. package/docs/tool-conformance.md +27 -2
  94. package/docs/tools.md +29 -2
  95. package/docs/web-tools.md +2 -2
  96. package/docs/wiki.md +6 -6
  97. package/docs/workflow-orchestration-primitives.md +24 -0
  98. package/docs/workflows.md +70 -9
  99. package/docs/working-and-semantic-memory.md +53 -5
  100. package/package.json +10 -30
  101. package/templates/README.md +23 -0
  102. package/templates/deep-research/README.md.tmpl +47 -0
  103. package/templates/deep-research/env.example.tmpl +12 -0
  104. package/templates/deep-research/gitignore.tmpl +7 -0
  105. package/templates/deep-research/manifest.json +12 -0
  106. package/templates/deep-research/package.json.tmpl +23 -0
  107. package/templates/deep-research/src/agent.ts.tmpl +81 -0
  108. package/templates/deep-research/src/index.ts.tmpl +53 -0
  109. package/templates/deep-research/src/tests/research.test.ts.tmpl +114 -0
  110. package/templates/deep-research/src/tools.ts.tmpl +86 -0
  111. package/templates/deep-research/src/types.ts.tmpl +45 -0
  112. package/templates/deep-research/src/workflow.ts.tmpl +156 -0
  113. package/templates/deep-research/tsconfig.json.tmpl +15 -0
  114. package/templates/init/manifest.json +5 -0
  115. package/templates/init/package.json.tmpl +2 -1
  116. package/templates/init/providers.json +16 -16
@@ -0,0 +1,213 @@
1
+ # Documents, spreadsheets, and presentations (`@arnilo/prism-office/documents`)
2
+
3
+ ## What it does
4
+
5
+ The `@arnilo/prism-office/documents` package provides specification-compliant, AI-native OpenXML document generation, parsing, patching, and bounded preview rendering for Microsoft Word (`.docx`), Microsoft Excel (`.xlsx`), and Microsoft PowerPoint (`.pptx`) artifacts.
6
+
7
+ It operates on a canonical, typed abstract syntax tree (AST) called the **Prism Document Model** (`DocModel`, `SheetModel`, `DeckModel`):
8
+ - **Pure in-memory doctrine**: Functions accept `Uint8Array` container buffers or typed model objects and emit `Uint8Array` buffers or JSON models. Zero filesystem reads, zero network I/O, zero `process.env` lookups, and zero child process spawns.
9
+ - **Draft-07 JSON Schema validation & slicing**: Full runtime structural validation with transitive closure slicing (`getDocumentModelSchema`) allowing LLM tools and agent prompts to extract minimal, self-contained sub-schemas (e.g. `doc.paragraph`, `doc.table`).
10
+ - **Bidirectional round-trip fidelity**: Prism-generated documents parse back into structurally equivalent models verified against a per-kind equality specification.
11
+ - **Typed model patch engine**: Immutably applies `set`, `insert`, `remove`, and `move` operations to document blocks, worksheet cells, and presentation slides with schema re-validation and an interactive `createPatchHistory` undo/redo stack.
12
+ - **Framework-neutral preview blocks & bounded HTML**: Generates structured snapshots (`PreviewBlock[]`) for native desktop/web UI grids and outline trees, as well as safe, sanitize-by-construction HTML fragments (`renderPreviewHtml`) guaranteed to contain no executable scripts, no active pseudo-protocols, and no external hyperlinks.
13
+ - **Boundary text redaction**: Pluggable `SecretRedactor` hook to sanitize extracted text content (paragraphs, cells, notes, tables) at the parse boundary before models are returned.
14
+ - **Dependency-free telemetry seam**: Zero-overhead `DocumentsTelemetry` hook for OpenTelemetry-compatible span tracing without leaking document or chunk text into telemetry collectors.
15
+
16
+ ## When to use it
17
+
18
+ Use `@arnilo/prism-office/documents` whenever autonomous agents, coding assistants, workflow orchestrators, or enterprise applications need to:
19
+ 1. Synthesize professional DOCX reports, financial XLSX spreadsheets, or PPTX presentation decks from structured LLM outputs.
20
+ 2. Ingest existing OOXML artifacts into a structured, validated document model for analysis or automated summarization.
21
+ 3. Perform atomic, validated updates or localized edits to documents using typed patch operations.
22
+ 4. Render safe, bounded HTML previews or framework-neutral outline and grid snapshots in web and desktop hosts.
23
+
24
+ Do **not** use this package for collaborative real-time editing (OT/CRDT), macro execution, or in-memory spreadsheet formula evaluation (formulas are preserved verbatim as `{ formula, cachedValue? }` read-only pairs).
25
+
26
+ ## Inputs / request
27
+
28
+ ### Generation & Parsing Functions
29
+
30
+ | Function | Signature | Description |
31
+ | --- | --- | --- |
32
+ | `generateDocument` | `(model: DocumentModel, options: GenerateDocumentOptions) => Promise<GenerateDocumentResult>` | Translates a typed model into spec-compliant OOXML binary bytes (PK zip container) with a SHA-256 content hash. |
33
+ | `parseDocument` | `(bytes: Uint8Array, options: ParseDocumentOptions) => Promise<DocumentModel>` | Verifies PK zip signature, enforces caps, translates OOXML parts, applies optional redaction, and returns a validated model. |
34
+ | `patchDocument` | `(model: DocumentModel, patches: readonly DocumentPatch[], options?: PatchDocumentOptions) => DocumentModel` | Clones the model, applies typed structural patch operations, and validates the resulting model against Draft-07 schemas. |
35
+ | `createPatchHistory` | `(initialModel: DocumentModel) => PatchHistory` | Creates an interactive undo/redo history manager for host editing workflows. |
36
+ | `renderPreviewBlocks` | `(model: DocumentModel, options?: PreviewBlocksOptions) => PreviewBlock[]` | Emits framework-neutral structured blocks (document outlines, bounded sheet grid chunks, slide summaries). |
37
+ | `renderPreviewHtml` | `(model: DocumentModel, options?: PreviewHtmlOptions) => string` | Emits safe, bounded HTML fragments with all entities escaped and external URLs neutralized. |
38
+ | `getDocumentModelSchema`| `(options: GetDocumentModelSchemaOptions) => Record<string, unknown>` | Retrieves full Draft-07 JSON Schema or a self-contained sliced sub-schema with resolved `$defs`. |
39
+ | `validateDocumentModel`| `(model: unknown) => asserts model is DocumentModel` | Validates arbitrary JSON objects against Draft-07 document schemas and structural invariants. |
40
+
41
+ ### Capacity Limits and Defaults
42
+
43
+ Caps are strictly enforced in memory before compute-intensive translation or parsing:
44
+
45
+ | Cap | Default | Hard Ceiling | Target |
46
+ | --- | --- | --- | --- |
47
+ | `maxBytes` | 32 MiB (`33,554,432`) | 512 MiB (`536,870,912`) | Binary input buffer & generated OOXML bytes |
48
+ | `maxBlocks` | 10,000 | 50,000 | Total blocks in a `DocModel` |
49
+ | `maxSheets` | 50 | 250 | Worksheets in a `SheetModel` |
50
+ | `maxRows` | 100,000 | 1,000,000 | Rows per worksheet |
51
+ | `maxColumns` | 1,000 | 16,384 | Columns per worksheet |
52
+ | `maxCells` | 1,000,000 | 10,000,000 | Total cells across all worksheets |
53
+ | `maxSlides` | 500 | 2,000 | Total slides in a `DeckModel` |
54
+ | `maxParagraphs` | 10,000 | 50,000 | Total paragraphs in a document |
55
+ | `maxRuns` | 50,000 | 250,000 | Total formatted text runs in a document |
56
+
57
+ ## Outputs / response / events
58
+
59
+ ### Error Hierarchy
60
+
61
+ All package operations throw typed exceptions derived from `DocumentsError`:
62
+
63
+ | Error Class | Error Code | Trigger Condition |
64
+ | --- | --- | --- |
65
+ | `DocumentsValidationError` | `ERR_PRISM_DOCUMENTS_INVALID_MODEL` | Input JSON fails Draft-07 schema validation, has dimension mismatches, or invalid decimal string syntax. |
66
+ | `DocumentsCapExceededError` | `ERR_PRISM_DOCUMENTS_CAP_EXCEEDED` | Document exceeds byte size, block count, cell count, or slide count caps. |
67
+ | `DocumentsParseError` | `ERR_PRISM_DOCUMENTS_PARSE_FAILED` | Input buffer lacks PK zip signature, is corrupted, or fails OOXML part parsing. |
68
+ | `DocumentsFormatError` | `ERR_PRISM_DOCUMENTS_UNSUPPORTED_FORMAT` | Format mismatch (e.g. attempting to generate PPTX from a `DocModel`). |
69
+ | `DocumentsPatchError` | `ERR_PRISM_DOCUMENTS_PATCH_FAILED` | Out-of-bounds index target, unknown patch operation, or invalid patch structure. |
70
+
71
+ ## Request/response example
72
+
73
+ ### JSON Schema Document Models
74
+
75
+ ```json
76
+ {
77
+ "kind": "sheet",
78
+ "modelVersion": 1,
79
+ "title": "Q3 Financial Summary",
80
+ "sheets": [
81
+ {
82
+ "name": "Revenue",
83
+ "columnWidths": [{ "column": 0, "width": 25 }, { "column": 1, "width": 15 }],
84
+ "frozenPanes": { "rows": 1, "columns": 0 },
85
+ "cells": [
86
+ ["Department", "Operating Budget", "Actual Expense"],
87
+ ["Engineering", { "type": "decimal", "value": "1500000.00" }, { "type": "decimal", "value": "1420000.50" }],
88
+ ["Operations", { "type": "decimal", "value": "450000.00" }, { "type": "decimal", "value": "435000.00" }],
89
+ ["Total", { "formula": "=SUM(B2:B3)", "cachedValue": 1950000 }, { "formula": "=SUM(C2:C3)", "cachedValue": 1855000.5 }]
90
+ ]
91
+ }
92
+ ]
93
+ }
94
+ ```
95
+
96
+ ## Implementation example
97
+
98
+ ```ts
99
+ import {
100
+ generateDocument,
101
+ parseDocument,
102
+ patchDocument,
103
+ createPatchHistory,
104
+ renderPreviewBlocks,
105
+ renderPreviewHtml,
106
+ type DocModel,
107
+ } from "@arnilo/prism-office/documents";
108
+
109
+ // 1. Define typed document model
110
+ const doc: DocModel = {
111
+ kind: "doc",
112
+ modelVersion: 1,
113
+ title: "Architecture Review",
114
+ blocks: [
115
+ { type: "heading", level: 1, text: "System Architecture" },
116
+ {
117
+ type: "paragraph",
118
+ runs: [
119
+ { text: "Prism operates purely in memory with ", bold: false },
120
+ { text: "zero-trust security boundaries.", bold: true, italic: true },
121
+ ],
122
+ },
123
+ {
124
+ type: "table",
125
+ rows: 2,
126
+ columns: 2,
127
+ cells: [
128
+ ["Module", "Latency"],
129
+ ["Parser", "12ms"],
130
+ ],
131
+ },
132
+ ],
133
+ };
134
+
135
+ // 2. Generate in-memory DOCX binary with SHA-256 contentHash
136
+ const { bytes, contentHash } = await generateDocument(doc, { format: "docx" });
137
+ console.log(`Generated DOCX (${bytes.byteLength} bytes, SHA-256: ${contentHash})`);
138
+
139
+ // 3. Parse OOXML bytes back to a validated model
140
+ const parsed = await parseDocument(bytes, { kind: "doc" });
141
+
142
+ // 4. Apply typed model patches
143
+ const patched = patchDocument(parsed, [
144
+ { op: "set", target: { block: 0 }, block: { type: "heading", level: 1, text: "Executive Summary" } },
145
+ { op: "insert", target: { afterBlock: 0 }, block: { type: "paragraph", text: "New introduction." } },
146
+ ]);
147
+
148
+ // 5. Interactive Undo/Redo editing
149
+ const history = createPatchHistory(patched);
150
+ history.apply([{ op: "set", target: { title: true }, value: "Updated Review" }]);
151
+ console.log(history.canUndo()); // true
152
+ const restored = history.undo(); // restored to "Executive Summary" state
153
+
154
+ // 6. Generate structured preview blocks & safe HTML
155
+ const blocks = renderPreviewBlocks(restored);
156
+ const htmlSnippet = renderPreviewHtml(restored, { maxHtmlBytes: 256 * 1024 });
157
+ ```
158
+
159
+ ## Extension and configuration notes
160
+
161
+ ### Sub-package Pinning
162
+ To avoid pulling in CLI frameworks or extraneous dependencies, `@arnilo/prism-office/documents` directly pins the exact underlying modular packages:
163
+ - `@office-open/docx@0.12.3`
164
+ - `@office-open/xlsx@0.12.3`
165
+ - `@office-open/pptx@0.12.3`
166
+
167
+ ### Draft-07 JSON Schema Slicing
168
+ For AI agent tool generation where token budgets are constrained, `getDocumentModelSchema` provides closure slicing:
169
+ ```ts
170
+ // Returns only TableBlock, CellValue, and their transitively required definitions in $defs
171
+ const tableSchema = getDocumentModelSchema({ kind: "doc", slice: "table" });
172
+ ```
173
+
174
+ ### Text Redaction Hook (`SecretRedactor`)
175
+ When parsing untrusted or sensitive OOXML containers, hosts can inject a redaction hook to scrub PII or secrets before the document model enters memory:
176
+ ```ts
177
+ const sanitizedModel = await parseDocument(rawBytes, {
178
+ kind: "doc",
179
+ redactor: {
180
+ redact: (text: string) => text.replace(/\b\d{3}-\d{2}-\d{4}\b/g, "[REDACTED-SSN]"),
181
+ },
182
+ });
183
+ ```
184
+
185
+ ### Telemetry Seam
186
+ The dependency-free `DocumentsTelemetry` seam allows optional OpenTelemetry instrumentation without adding runtime telemetry dependencies:
187
+ ```ts
188
+ const telemetry: DocumentsTelemetry = {
189
+ startSpan(name, attributes) {
190
+ // Maps to tracer.startSpan with allow-listed metadata (kind, format, bytes, counts)
191
+ // Model content and document text are NEVER passed to telemetry spans.
192
+ return activeSpan;
193
+ },
194
+ };
195
+ ```
196
+
197
+ ### Decimal Fidelity Ceiling
198
+ Financial worksheets often require exact decimal representations that JavaScript 64-bit binary floating-point numbers cannot represent without precision loss. `@arnilo/prism-office/documents` supports canonical string decimals (`{ type: "decimal", value: "1500000.00" }`), preserving exact numerical formatting without IEEE 754 precision artifacts.
199
+
200
+ ## Security and performance notes
201
+
202
+ - **Pure In-Memory Operation**: No temporary files, no shell execution, no binary spawning, and zero network sockets.
203
+ - **ZIP Signature Gating**: Buffers must begin with PK zip container signatures (`0x50, 0x4B, 0x03, 0x04`). Extension-based type inference is strictly prohibited.
204
+ - **Fail-Closed Caps**: Input size and element count caps are evaluated before entering XML translation passes, preventing zip-bomb and decompression amplification attacks.
205
+ - **Sanitize-by-Construction HTML**: `renderPreviewHtml` strictly entity-encodes all text fields, neutralizes dangerous protocols (`javascript:`, `http://`, `https://`), strips raw script/image tags, and caps output size to prevent DOM-based XSS and memory exhaustion.
206
+ - **Performance Budget**: Warm generation of 200-block documents completes in under 15 ms; parse and round-trip equality checks complete in under 100 ms.
207
+
208
+ ## Related APIs
209
+
210
+ - [`@arnilo/prism-document-reader`](./document-reader.md): Bounded literal text extraction from PDF and DOCX documents for coding agent tools.
211
+ - [`@arnilo/prism-work-tools`](./work-tools.md): Microsoft 365 and Google Workspace identity-scoped connectors.
212
+ - [`@arnilo/prism-coding-agent`](./coding-agent-tools.md): Coding tools and file operations.
213
+ - [`@arnilo/prism-observability-opentelemetry`](./observability.md): OpenTelemetry instrumentation and trace adapters.
@@ -19,6 +19,7 @@ Use this package when a host needs offline quality checks or sampled live scorin
19
19
  | `createMemoryEvaluationStore` | optional seed records |
20
20
  | `appendEvaluationFeedback` | `RunFeedbackStore`, `EvaluationStore`, feedback fields, and 1–64 known evaluation IDs |
21
21
  | `createPersistenceTraceResolver` | explicit `ProductionPersistenceStore`, exact session/run/ownership, page/byte bounds |
22
+ | `datasetFromRuns` | `runIds` and/or `sessionIds`, existing dataset, `ProductionPersistenceStore`, ownership, redactor/secrets, optional `toItem` |
22
23
  | `createModelJudge` | host judge callback, stable rubric/version, timeout/attempt/output bounds |
23
24
  | `runComparison` | immutable dataset, 2–8 named candidates by default, pairwise scorers |
24
25
  | `assertEvaluationThreshold` / `serializeEvaluationReport` | mean/failure/per-scorer gates and bounded redacted JSON |
@@ -32,6 +33,7 @@ Use this package when a host needs offline quality checks or sampled live scorin
32
33
  | `runExperiment` | `ExperimentReport` with stable item order, evaluations, and aggregates |
33
34
  | `EvaluationStore.query` | cursor-paginated, ownership-filtered page |
34
35
  | `appendEvaluationFeedback` | immutable `RunFeedbackRecord` containing only evaluation/scorer IDs |
36
+ | `datasetFromRuns` | `{ dataset, version, added, skipped }` — new immutable dataset version with one item per added run; skips carry reasons (missing run, ownership mismatch, empty output) |
35
37
 
36
38
  ## Request/response example
37
39
 
@@ -121,6 +123,7 @@ console.log(report.aggregate.meanScore, linked.evaluationIds);
121
123
  - Model judges are host callbacks, not providers: Prism passes rubric/version plus bounded target only—never credential resolvers, tools, or workspace. Defaults are one attempt, 30 seconds, and 16 KiB output; failures become redacted evaluation records.
122
124
  - Pairwise candidates are sorted by name, executed once per item, compared in stable item/pair/scorer order, and record ties/failures without choosing a winner. Candidate and scorer outputs have byte caps.
123
125
  - `assertEvaluationThreshold()` throws `ERR_PRISM_EVAL_THRESHOLD`; an uncaught error gives CI a non-zero exit. Keep model-judge/live gates credential-gated and outside the network-free default suite. `serializeEvaluationReport()` bounds/redacts checked-in artifacts.
126
+ - `datasetFromRuns` redacts before item construction and never stores an unredactable item; curated items are byte-capped (`ERR_PRISM_EVAL_CURATE`), ownership is verified exactly per run, and a failed append leaves the prior dataset version untouched.
124
127
 
125
128
  ## Trace, judge, comparison, and CI example
126
129
 
@@ -137,12 +140,43 @@ assertEvaluationThreshold(report, { minimumMean: 0.9, maximumFailures: 0 });
137
140
 
138
141
  `traceResolver` is explicit; no arbitrary run search occurs. `baseline`/`candidate` are host functions returning `AgentRunResult`. See `examples/evaluation-gate.ts` for a network-free gate and `examples/coding-browser-evaluation.ts` for coding/browser adversarial fixtures.
139
142
 
143
+ ## Curating datasets from production runs
144
+
145
+ Plan 043 adds `datasetFromRuns`: it turns recorded runs (production incident transcripts included) into dataset items in one call — resolve through `createPersistenceTraceResolver`, redact, map through an optional host `toItem`, and append as a **new immutable dataset version**. The prior version object is never mutated.
146
+
147
+ ```ts
148
+ import { datasetFromRuns, defineDataset } from "@arnilo/prism-evals";
149
+
150
+ const result = await datasetFromRuns({
151
+ runIds: ["run_9f2", "run_a71"],
152
+ dataset: defineDataset({ id: "support-regressions", items: [] }),
153
+ store: persistence,
154
+ ownership: { tenantId: "t1", userId: "u1" },
155
+ redactor,
156
+ toItem: (run) => ({
157
+ input: run.input,
158
+ expected: run.feedback?.metadata?.expected, // human-graded gold from the feedback seam
159
+ metadata: { graded: run.feedback?.rating },
160
+ }),
161
+ });
162
+ // result.added === 2, result.dataset.version === "2"
163
+ ```
164
+
165
+ Omit `toItem` and the default mapping is used: `input` = first user message, `expected` = the recorded feedback's `metadata.expected` when present, `output` = final assistant text under `metadata.output`.
166
+
167
+ - Session ids (`sessionIds`) expand to every run recorded under the session; bare `runIds` are located by one ownership-bounded scan of the run ledger (page/byte caps apply).
168
+ - Each resolved run maps to one item keyed by the run id. The default `toItem` carries the first user message as `input`; `expected` comes **only from the feedback seam** (`metadata.expected` of the latest human-graded `RunFeedbackRecord`) — never re-derived from untrusted outputs, and omitted entirely (not fabricated) when a run has no feedback. The recorded output rides `metadata.output` for provenance. A host `toItem` always wins and may return `undefined` to drop a run (`host filter` skip).
169
+ - Feedback costs one bounded, owner-scoped query per curation batch (`store.feedback`); records are read id-only per the feedback linkage contract — scorer payloads are never copied. A feedback query failure (e.g. scope contract mismatch) omits `expected` instead of aborting the batch.
170
+ - Every item field passes the host `redactor` after host mapping — fail closed: a redactor failure or an item over the frozen 4 MiB cap (`ERR_PRISM_EVAL_CURATE`) skips the run or aborts the append before anything is persisted. Cross-tenant runs are never readable (resolver ownership check → `ownership mismatch` skip).
171
+
172
+ Prompt versions can ride the same primitives: [`assertPromptPromotion`](prompt-registry.md#eval-gated-promotion) in `@arnilo/prism-prompts` resolves two prompt versions, runs them through `runComparison`, and returns a `promote`/`hold` verdict with per-scorer aggregates and a bounded report — never applying the change itself.
173
+
140
174
  ## Coding and browser adversarial evaluations (0.0.9)
141
175
 
142
176
  Release 0.0.9 ships curated network-free adversarial fixtures in package tests:
143
177
 
144
178
  - `@arnilo/prism-coding-agent` `eval-fixtures.test.ts`: safe native list vs shell, Git path/ref injection, dirty-tree rollback, unknown named-check failure, PR-handoff artifact completeness, and prompt-injection file content under read-only tools.
145
- - `@arnilo/prism-browser` `eval-fixtures.test.ts`: stale snapshot refs, side-effect approval, private/loopback/file deny, upload/download/screenshot policy, CSS/evaluate target rejection, and hostile accessible-name text.
179
+ - `browser` `eval-fixtures.test.ts`: stale snapshot refs, side-effect approval, private/loopback/file deny, upload/download/screenshot policy, CSS/evaluate target rejection, and hostile accessible-name text.
146
180
 
147
181
  Fixtures reuse `@arnilo/prism-evals` (`defineDataset` / `defineScorer` / `scoreRun` / `assertEvaluationThreshold` / `serializeEvaluationReport`). Optional SWE-bench-compatible or live-browser harnesses remain host adapters — they are not default dependencies or quality claims. Protected real Docker/Playwright gates stay env-gated (`PRISM_TEST_DOCKER_SANDBOX`, `PRISM_LIVE_PLAYWRIGHT`) and never enter `sdk:ready`.
148
182
 
package/docs/graft.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  ## What it does
4
4
 
5
- `@arnilo/prism-graft` is an optional package that wires [nanonets/graft](https://github.com/nanonets/graft) — a repository context-graph CLI (`graft/` directory, INDEX.md orientation, symbol-level wiring graph) — into Prism contribution contracts.
5
+ `@arnilo/prism-memory/graft` is an optional subpath that wires [nanonets/graft](https://github.com/nanonets/graft) — a repository context-graph CLI (`graft/` directory, INDEX.md orientation, symbol-level wiring graph) — into Prism contribution contracts.
6
6
 
7
7
  It registers six pull tools backed by the graft CLI (`--json`, argv-safe), a push-mode retrieval-pack context provider plus first-turn orientation injector carried on the `graft` skill, commands (`graft`, `graft-build`, `graft-check`, `graft-viz`), and an edit-watch middleware that computes blast radius after mutating tool calls. Import is inert; a missing graft CLI fails closed at `setup` with a bounded redacted error.
8
8
 
@@ -74,7 +74,7 @@ Tool call (pull):
74
74
  Status event:
75
75
 
76
76
  ```json
77
- { "type": "graft:status", "extension": "@arnilo/prism-graft", "metadata": { "fresh": true, "missing": 0, "stale": 2 } }
77
+ { "type": "graft:status", "extension": "@arnilo/prism-memory/graft", "metadata": { "fresh": true, "missing": 0, "stale": 2 } }
78
78
  ```
79
79
 
80
80
  ## Implementation example
@@ -83,7 +83,7 @@ See [`examples/graft-extension.ts`](../examples/graft-extension.ts) — network-
83
83
 
84
84
  ```ts
85
85
  import { createExtensionKernel, createMemorySessionStore } from "@arnilo/prism";
86
- import { createGraftExtension } from "@arnilo/prism-graft";
86
+ import { createGraftExtension } from "@arnilo/prism-memory/graft";
87
87
 
88
88
  const store = createMemorySessionStore();
89
89
  const kernel = createExtensionKernel({ errorPolicy: "throw" });
@@ -78,7 +78,7 @@ Guardrails are callbacks supplied by the host. Prism does not discover, load, re
78
78
 
79
79
  Optional `@arnilo/prism-policy` can record guardrail outcomes via `recordGuardrailDecision` (evidence refs only; see [Policy and audit](policy-and-audit.md)).
80
80
 
81
- Output buffering prevents blocked provider content from reaching subscribers, session entries, ledgers, parsers, delegation, or tools. Tool-output checks receive raw results but Prism discards blocked raw output before event, ledger, transcript, or MCP exposure. Redaction replaces exact known values only; it is not general secret detection. Parallel checks receive an abort signal, but callback code must honor it to stop in-flight work. Browser snapshots and page text from `@arnilo/prism-browser` are untrusted external content: never allow them to modify tools, permissions, credentials, or policy. Browser mutations still require host `ExecutionPolicy`/approval; prompt-injection text in a page cannot grant upload/download release.
81
+ Output buffering prevents blocked provider content from reaching subscribers, session entries, ledgers, parsers, delegation, or tools. Tool-output checks receive raw results but Prism discards blocked raw output before event, ledger, transcript, or MCP exposure. Redaction replaces exact known values only; it is not general secret detection. Parallel checks receive an abort signal, but callback code must honor it to stop in-flight work. Browser snapshots and page text from the `browser` subpath are untrusted external content: never allow them to modify tools, permissions, credentials, or policy. Browser mutations still require host `ExecutionPolicy`/approval; prompt-injection text in a page cannot grant upload/download release.
82
82
 
83
83
  ## Related APIs
84
84
 
@@ -149,10 +149,10 @@ Wire those values where they matter: provider adapters receive the resolved cred
149
149
  - AG-UI fields stay untrusted after schema validation. `input.project` returns host-selected messages/handoffs only; never merge state/tools/context/props into ownership, identity, permissions, provider options, or media policy. Apply Prism media SSRF/MIME bounds before resolution; output projectors are bounded allow-lists; interrupt edits deny rather than mutate persisted calls.
150
150
  - AG-UI MCP Apps requires negotiated `mcpApps`, exact proxy origin/auth, owned-run context, approval, one bridge, separate-origin sandbox (`allow-scripts allow-same-origin`), and no-wider CSP. Never execute HTML in host origin or retry a UI mutation; Task 4 adds recovery.
151
151
  - AG-UI A2A requires exact-origin verified client, host-owned task selection/correlation, explicit data/tool/A2UI projection, and reauthorized follow/cancel.
152
- - `@arnilo/prism-server` exposes no agent/workflow by default and requires `authorize()` for every matched operation. Derive complete tenant/account/user ownership from validated host identity, never request JSON. Workflow active identity and cancellation compare exact ownership; a tenant-only scope intentionally cannot cancel a checkpoint/run carrying account or user identity. The artifact review service (`createArtifactService`) requires authenticated identity + thread ownership on every attach/revise/compare/approve/reject/download, resolves concurrent reviewers via checkpoint CAS (no lost approvals), rejects local filesystem paths in `uri`/citations, redacts records before persist and on response, and serves downloads only through signed expiring links that are reauthorized against the token's ownership per request. When a blob store is wired (`bodies: ArtifactBodyStore`, 0.0.28), delivery links additionally resolve through `bodies.presign`; the reference `createS3ArtifactBodyStore` verifies ownership on every operation, verifies size/SHA-256/MIME on put and get (fail closed), refuses delete under legal hold (host `isHeld` callback), keeps credentials host-resolved, and never discloses bucket/path/key in errors, telemetry, or artifact records. Pass the current explicitly revised workflow definition so recursive hash mismatch fails before abort or durable mutation. Configure exact host/origin allow-lists where needed, wire redaction before execution, retain tool/workflow policy checks, and adapt the Web handler behind host TLS/rate limits. Disconnect abort is default; persistent reconnect/status belongs to durable workflow checkpoints, not an invented in-memory agent result cache.
152
+ - `@arnilo/prism-server` exposes no agent/workflow by default and requires `authorize()` for every matched operation. Derive complete tenant/account/user ownership from validated host identity, never request JSON. Workflow active identity and cancellation compare exact ownership; a tenant-only scope intentionally cannot cancel a checkpoint/run carrying account or user identity. The artifact review service (`createArtifactService`) requires authenticated identity + thread ownership on every attach/revise/compare/approve/reject/download, resolves concurrent reviewers via checkpoint CAS (no lost approvals), rejects local filesystem paths in `uri`/citations, redacts records before persist and on response, and serves downloads only through signed expiring links that are reauthorized against the token's ownership per request. When a blob store is wired (`bodies: ArtifactBodyStore`, 0.0.28), delivery links additionally resolve through `bodies.presign`; the reference `createS3ArtifactBodyStore` verifies ownership on every operation, verifies size/SHA-256/MIME on put and get (fail closed), refuses delete under legal hold (host `isHeld` callback), keeps credentials host-resolved, and never discloses bucket/path/key in errors, telemetry, or artifact records. Pass the current explicitly revised workflow definition so recursive hash mismatch fails before abort or durable mutation. Configure exact host/origin allow-lists where needed, wire redaction before execution, retain tool/workflow policy checks, and adapt the Web handler behind host TLS/rate limits. Disconnect abort is default; persistent reconnect/status belongs to durable workflow checkpoints, not an invented in-memory agent result cache. Outbound lifecycle webhooks (`createWebhookNotifier`, 0.3.2) share the same boundary posture: host-registered public HTTPS (or explicitly opted-in loopback HTTP) targets only, private/metadata literals rejected at registration, every attempt DNS-pinned and redirect-free through core `pinnedFetch`, redaction before HMAC signing, and the key held by the host only. Pass a known-secret `SecretRedactor`.
153
153
  - Coding tools from `@arnilo/prism-coding-agent` accept an optional `ExecutionPolicy` checked inside each tool before side effects; shared policy propagation includes `createReadOnlyTools()`. They enforce finite text-scan/image/edit/write/shell limits, repository list/search depth/entry/match/scan/time caps, structured Git path/ref/message/output/patch/worktree caps, named-check concurrency/output caps, a 600-second default shell wall time, and a 64 MiB default total-output ceiling. Opt-in `createGitTools()` uses argument arrays with hooks/credential prompts/external diff disabled, requires host `commitIdentity` for commits, and never pushes or opens PRs. Successful truncated shell output leaves a host-owned exclusive `0600` temp file; delete `metadata.fullOutputPath` after use. Error/abort/timeout/overflow removes unpublished spills. Custom read/edit/shell/repository backends must honor supplied caps/signals. Use `@arnilo/prism-coding-security` for path roots, command rules, identity-scoped approval caching, required `workspaceMode` on `createSandboxCodingComposition()` / `createSandboxCodingTools()`, and the optional `createDockerSandbox()` reference adapter. **Host mode is never contained execution** (every isolation capability false). Sandbox mode reports isolation only from validated adapter capability metadata: `composition.capabilities` carries the frozen `SandboxCapabilities` object (`workspaceCoherent`, `filesystemIsolated`, `networkIsolated`, `processIsolated`, `privilegeIsolated`, `egressRestricted`); the deprecated `containmentClaim` is a conservative projection and must never be used alone. Authorize security-sensitive actions from the individual capabilities the policy actually needs — e.g. require `filesystemIsolated` before hosting untrusted coding tasks, and `egressRestricted` before any network-capable run. Mixed wiring requires `allowMixedWorkspaceWiring` and still reports no isolation. Limits alone are not containment: construct the Docker adapter (absolute CLI, digest-pinned image, network none by default) or an equivalent host sandbox before treating coding execution as production-safe. Docker daemon/image trust, egress firewall/proxy, and artifact retention remain host-owned.
154
154
  - Allow-list egress (0.0.26, `@arnilo/prism-coding-security`): `createEgressPolicy()` is deny-all with exact host/port/protocol rules and frozen `npm-registry`/`github` presets; `createAllowListEgressProxy()` is an HTTP forward proxy + CONNECT tunnel that pins DNS answers and verifies the connected address (rebinding defense), denies private/link-local/metadata ranges unless a rule opts in, re-validates every redirect hop against policy, and cuts oversized/slow transfers at frozen byte/time caps. TLS passes through without interception. Every allow/deny writes an audit record with no secrets. The proxy is inert until `start()`; `reloadPolicy()` is the only rule change path. `composeEgressSandboxNetwork(proxy.attestation(), name)` records validated attestation as `prism.egress.*` container labels — evidence, not enforcement: the host must restrict the Docker network so the proxy is the only reachable path, and `denyDirectEgress: true` is a claim the host makes true by topology. The proxy is not a firewall and cannot stop a container whose network reaches the internet directly.
155
- - Optional `@arnilo/prism-browser` requires a host-supplied Playwright Browser (`playwright-core@1.61.0` peer). Import is inert. One non-persistent context belongs to one run; actions serialize; refs are snapshot-scoped; CSS/evaluate/CDP/persistent profiles are denied. Context routing + `serviceWorkers: "block"` deny file/data/blob/devtools/private/loopback by default and require contained-proxy attestation for external egress (Playwright routing is defense in depth, not DNS containment). Uploads are realpath-rooted; downloads quarantine with hash/MIME until host `approveRelease`; screenshots return bounded `ImageContent`. Observation vs mutation/high-impact actions map to `ExecutionPolicy`. Treat snapshot/page text as untrusted external content. Close contexts with `browser_close` or `manager.closeRun(runId)` on abort/terminal. Browser control endpoint, binary/image pin, and real egress firewall/proxy remain host-owned. Shared sandbox: `createSharedSandboxBrowserOptions()` + `assertBrowserSandboxNetwork()`.
155
+ - Optional `@arnilo/prism-web-tools/browser` requires a host-supplied Playwright Browser (`playwright-core@1.61.0` peer). Import is inert. One non-persistent context belongs to one run; actions serialize; refs are snapshot-scoped; CSS/evaluate/CDP/persistent profiles are denied. Context routing + `serviceWorkers: "block"` deny file/data/blob/devtools/private/loopback by default and require contained-proxy attestation for external egress (Playwright routing is defense in depth, not DNS containment). Uploads are realpath-rooted; downloads quarantine with hash/MIME until host `approveRelease`; screenshots return bounded `ImageContent`. Observation vs mutation/high-impact actions map to `ExecutionPolicy`. Treat snapshot/page text as untrusted external content. Close contexts with `browser_close` or `manager.closeRun(runId)` on abort/terminal. Browser control endpoint, binary/image pin, and real egress firewall/proxy remain host-owned. Shared sandbox: `createSharedSandboxBrowserOptions()` + `assertBrowserSandboxNetwork()`.
156
156
  - Browser verified-state checkpoints (0.0.14, `createBrowserCheckpointLedger()`) store URL + domain-state hash + host data refs only — never serialized browser internals (cookies/storage/contexts). After any resume/interruption the ledger fails closed (`assertVerifiedBeforeSideEffect`) until the host reloads + verifies, so side effects never replay on stale state.
157
157
  - Device adapters (0.0.14, `resolveDevicePolicy`/`assertDeviceAdmit`) are deny-by-default: admission fails closed without explicit `enabled`, an explicit sandbox, approval (when required), an under-budget session count, and shared `RunLimits`. Stream chunks over the frozen cap are dropped with a marker; telemetry is redacted before emit/persist. No vendor voice/desktop package ships in 0.0.14 (demand-gated 0.1.x); device adapters cannot broaden consent/memory/network/file/browser/connector/tool permissions (gate 8).
158
158
  - Optional `@arnilo/prism-wiki` tools treat agent-supplied input as untrusted at the first-party `.wiki/` filesystem boundary. `wiki_read_page` enforces lexical containment (`path.relative` with separator-aware `..`/absolute checks) plus `fs.realpath` containment for the wiki root and every successfully read file, so sibling-prefix (`.wiki-evil`), `..`, absolute, alternate-separator, and symlink escapes are denied before content is returned; missing contained pages report `found: false` while denied paths throw an access-denied error (never mapped to not-found). `wiki_record_insight` rejects empty titles/content, caps titles at 200 characters and content at 65,536 bytes, and collapses control characters and newlines in titles to single-line display text before any page/frontmatter/index/log write, so titles cannot inject Markdown headings, index entries, or log entries; slugs are allow-listed to `[a-z0-9-_]` with a non-empty fallback. See [LLM Wiki](wiki.md).
@@ -211,7 +211,8 @@ PostgreSQL TLS/network policy, MCP endpoint trust/credentials and egress policy
211
211
  ### 0.1.0 security evidence (plan 012 Task 6)
212
212
 
213
213
  - **Audit policy.** `npm audit --audit-level=moderate` is enforced in both `security.yml` and the `release.yml` supply-chain job (freeze-manifest `releasePolicy.auditLevelTarget`); recorded 0.1.0 tree: 0 vulnerabilities at every severity (317 locked dependencies, MCP SDK at the 1.30.0 fix baseline).
214
- - **Named threat-suites leg.** `npm run security:threat-suites` aggregates the Phase 8–11 conformance suites (durable-loop/HITL approval, coding sandbox/egress/forge, ACP protocol, OIDC/OPA/MCP-OAuth/OpenAPI/artifact) into one named 0.1.0 security evidence leg — same scripts as `npm test`, no rewrite; the Phase 7 tenant-isolation suite is its protected counterpart under `npm run test:postgres` (missing `PRISM_TEST_POSTGRES_URL` is a named blocked gate).
214
+ - **Named threat-suites leg.** `npm run security:threat-suites` aggregates the Phase 8–11 conformance suites (durable-loop/HITL approval, coding sandbox/egress/forge, ACP protocol, OIDC/OPA/MCP-OAuth/OpenAPI/artifact) into one named 0.1.0 security evidence leg — same scripts as `npm test`, no rewrite; the Phase 7 tenant-isolation suite is its protected counterpart under `npm run test:postgres` (missing `PRISM_TEST_POSTGRES_URL` is a named blocked gate). The dev-inspector leg (`scripts/phase40-security.test.mjs`, plan 040 Task 5) exercises the `@arnilo/prism-dev` local playground through built public entrypoints: D1 non-loopback binds refuse before any listener exists (`ERR_PRISM_DEV_REMOTE_BIND`) and loopback is the only bindable surface; D2 the host redactor scrubs secret literals from server-rendered replay payloads; D3 replay selectors outside the `resolveRun` seam's ownership fail closed 404 and replay without a durable event_source is a documented 404, never a re-execution; D4 unknown decision outcome discriminants reject `400` without consuming a run version or executing the tool, and a valid `allow_once` applies exactly once.
215
+ - **Dev-surface boundary (plan 040 Task 5).** The `@arnilo/prism-dev` inspector is a developer-time tool: loopback-only by default, non-loopback requires an explicit programmatic `remoteAuthorize` callback plus a real authorizer (the CLI/bin offers no remote-bind flag at all), and it must never be the production API boundary (that stays `@arnilo/prism-server` under host authorization). It carries no credential storage — the host agent config owns credentials and the inspector never reads environment secrets; rendered tool args/results pass the host redactor on both the live and replay paths, and HITL decision resume composes the core `createAgentRunLifecycle` validation (fail-closed on unknown discriminants/versions/runs).
215
216
  - **Supply-chain negative fixtures.** `scripts/release-gate.test.mjs` verifies the tarball deny list rejects tampered content (plans/reviews/maps/tests), unexpected file types and credential material (native binaries, `.pem`/`.key`/`.p12`), and that a provenance flag suppressed in CI is detectable in the `release.mjs` publish dry-run arguments (`--provenance` mandatory under `GITHUB_ACTIONS`, never claimed on local OIDC-less publishes).
216
217
  - **Mandatory gate stack.** CodeQL/SAST, PR dependency review (fail on high), secret scan (source + unpacked tarballs), SPDX SBOM + license policy, tarball allow/deny content checks, and provenance (npm OIDC + GitHub build attestations on tarballs and SBOM) all run in `security.yml`/`release.yml`; evidence for the 0.1.0 tree is recorded in [0.1.0 readiness](0.1.0-readiness.md).
217
218
  - **CodeQL query suite.** `.github/codeql/codeql-config.yml` selects the `security-extended` suite for `javascript-typescript` (with the default suite) on push/PR/schedule in `security.yml` (10-minute job bound; measured runtime ~3m22s on the audited SHA, last successful main run `33059128198`). The ignore list covers only generated `dist`, `node_modules`, and release/security artifact directories — first-party packages, threat suites, and fixtures that ship or execute are always scanned, so new alerts enter the same plan-038 ledger/remediation loop (config + guardrails asserted in `scripts/phase38-codeql-regression.test.mjs`). Local Task 6 gates (typecheck/lint/format/threat suites/audit/secret scan/SBOM) pass on the remediations; GitHub `state=open` stays non-zero until those remediations are the analyzed head. Groups G (`js/insufficient-password-hash` on RFC 7636 S256) and H (`js/incomplete-url-substring-sanitization` on a negative docs assertion) are maintainer-reviewed false positives queued for narrow dismissal after that analyze, not code changes.
@@ -2,7 +2,7 @@
2
2
 
3
3
  ## What it does
4
4
 
5
- `@arnilo/prism-impeccable` is an optional package that wires a host-supplied
5
+ `@arnilo/prism-coding-tools/impeccable` is an optional package that wires a host-supplied
6
6
  [Impeccable](https://github.com/pbakaus/impeccable) `SKILL.md` into Prism skill
7
7
  and command registries.
8
8
 
@@ -56,7 +56,7 @@ No instruction injector. No session persistence. No 23 Prism-native craft/polish
56
56
  ## Implementation example
57
57
 
58
58
  ```ts
59
- import { createImpeccableExtension } from "@arnilo/prism-impeccable";
59
+ import { createImpeccableExtension } from "@arnilo/prism-coding-tools/impeccable";
60
60
  import {
61
61
  createExtensionKernel,
62
62
  createLoadSkillTool,