@arnilo/prism 0.6.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +79 -5
- package/README.md +12 -11
- package/dist/agent-approval.d.ts +4 -0
- package/dist/agent-approval.js +5 -1
- package/dist/agent-definitions.js +1 -0
- package/dist/agent-run-lifecycle.js +39 -4
- package/dist/agent-run-state.d.ts +18 -0
- package/dist/agent-run-state.js +39 -9
- package/dist/agent-session/helpers.js +6 -1
- package/dist/agent-session/session/assemble.js +159 -7
- package/dist/agent-session/session/persist.d.ts +16 -0
- package/dist/agent-session/session/persist.js +64 -4
- package/dist/agent-session/session/provider-round.d.ts +3 -3
- package/dist/agent-session/session/provider-round.js +12 -6
- package/dist/agent-session/session/tool-round.js +5 -1
- package/dist/agent-session/session/types.d.ts +22 -1
- package/dist/agent-session/session.d.ts +16 -0
- package/dist/agent-session/session.js +42 -3
- package/dist/artifacts.d.ts +39 -1
- package/dist/artifacts.js +73 -0
- package/dist/attention-compiler.d.ts +121 -0
- package/dist/attention-compiler.js +479 -0
- package/dist/checkpoints.js +7 -11
- package/dist/cli-init.js +20 -6
- package/dist/context-budget.d.ts +20 -1
- package/dist/context-budget.js +10 -1
- package/dist/contracts-core/agent.d.ts +7 -0
- package/dist/contracts-core/attention.d.ts +66 -0
- package/dist/contracts-core/attention.js +2 -0
- package/dist/contracts-core/compaction.d.ts +59 -0
- package/dist/contracts-core/compaction.js +77 -1
- package/dist/contracts-core/content.d.ts +5 -0
- package/dist/contracts-core/loop.d.ts +42 -0
- package/dist/contracts-core/provider.d.ts +4 -0
- package/dist/contracts-core/run-limits.d.ts +2 -0
- package/dist/contracts-core.d.ts +1 -0
- package/dist/contracts-core.js +1 -0
- package/dist/contracts-protocol.d.ts +44 -3
- package/dist/contracts-run-state.d.ts +32 -5
- package/dist/evidence-grounding.d.ts +29 -0
- package/dist/evidence-grounding.js +162 -0
- package/dist/host-composition.d.ts +91 -0
- package/dist/host-composition.js +279 -0
- package/dist/index.d.ts +13 -6
- package/dist/index.js +7 -4
- package/dist/input.d.ts +13 -1
- package/dist/input.js +40 -1
- package/dist/provider-events.d.ts +3 -1
- package/dist/provider-events.js +2 -2
- package/dist/providers/transport.d.ts +3 -1
- package/dist/providers/transport.js +36 -0
- package/dist/redaction.js +18 -2
- package/dist/run-bundle.d.ts +89 -0
- package/dist/run-bundle.js +149 -0
- package/dist/secure-agent.d.ts +2 -0
- package/dist/secure-agent.js +6 -1
- package/dist/testing/state-concurrency-conformance.js +5 -12
- package/dist/tool-result-fold.d.ts +12 -0
- package/dist/tool-result-fold.js +13 -6
- package/dist/tools.d.ts +10 -0
- package/dist/tools.js +41 -0
- package/docs/acp-agent.md +42 -11
- package/docs/acp.md +2 -1
- package/docs/ag-ui.md +10 -3
- package/docs/agent-definitions.md +9 -1
- package/docs/agent-events.md +4 -1
- package/docs/agent-loops.md +33 -0
- package/docs/agent-session-runtime.md +8 -7
- package/docs/attention-compiler.md +272 -0
- package/docs/cli-rpc.md +4 -2
- package/docs/coding-agent-tools.md +1 -1
- package/docs/coding-security.md +6 -3
- package/docs/coding-tools.md +0 -1
- package/docs/coding-workspaces.md +22 -0
- package/docs/compaction-and-retry.md +36 -4
- package/docs/compaction-observational-memory.md +63 -10
- package/docs/connected-apps.md +116 -0
- package/docs/context-and-skills.md +17 -2
- package/docs/conversations.md +1 -1
- package/docs/core.md +1 -1
- package/docs/dev-inspector.md +4 -0
- package/docs/device-adapters.md +1 -0
- package/docs/diagrams.md +6 -6
- package/docs/document-reader.md +18 -10
- package/docs/documents.md +40 -11
- package/docs/durable-runs.md +87 -0
- package/docs/enterprise-postgres-state.md +6 -2
- package/docs/evaluations.md +168 -4
- package/docs/execution-timeline.md +186 -0
- package/docs/guardrails.md +33 -0
- package/docs/history/0.7.0-primitive-review.md +254 -0
- package/docs/history/079-messaging-primitive-review.md +391 -0
- package/docs/history/080-messaging-followon-primitive-review.md +234 -0
- package/docs/history/081-connected-apps-primitive-review.md +74 -0
- package/docs/history/083-prism-work-primitive-review.md +84 -0
- package/docs/history/084-primitive-review.md +96 -0
- package/docs/history/085-honesty-and-cut-primitive-review.md +91 -0
- package/docs/history/README.md +5 -0
- package/docs/history/migration-0.0.md +2 -2
- package/docs/history/release-handoffs.md +75 -1
- package/docs/host-compositions.md +149 -0
- package/docs/host-security.md +2 -2
- package/docs/hosted-sandboxes.md +94 -0
- package/docs/index.md +82 -45
- package/docs/input-and-prompt-assembly.md +1 -0
- package/docs/knowledge-sync.md +84 -0
- package/docs/language-intelligence.md +1 -1
- package/docs/live-testing.md +8 -3
- package/docs/mcp-tools.md +3 -1
- package/docs/memory-fabric.md +416 -0
- package/docs/messaging-channel-operations.md +166 -0
- package/docs/messaging-channels.md +150 -0
- package/docs/migrate-to-0.5.md +1 -1
- package/docs/migrate-to-0.6.md +1 -0
- package/docs/migrate-to-0.7.md +345 -0
- package/docs/migrate-to-0.8.md +124 -0
- package/docs/migration.md +43 -1
- package/docs/model-registry.md +12 -2
- package/docs/model-routing.md +79 -4
- package/docs/multi-agent-patterns.md +20 -6
- package/docs/observability.md +52 -1
- package/docs/openapi-tools.md +1 -1
- package/docs/operations.md +14 -4
- package/docs/options-index.md +47 -3
- package/docs/peer-dependencies.md +12 -10
- package/docs/postgres-persistence.md +1 -1
- package/docs/process-sessions.md +3 -1
- package/docs/prompt-registry.md +1 -1
- package/docs/provider-caching.md +4 -2
- package/docs/provider-conformance.md +1 -1
- package/docs/provider-layer.md +2 -2
- package/docs/provider-packages.md +22 -22
- package/docs/providers/bedrock.md +71 -7
- package/docs/providers/neuralwatt.md +5 -1
- package/docs/providers/openai.md +1 -1
- package/docs/rag.md +24 -8
- package/docs/realtime-voice.md +87 -0
- package/docs/release-and-install.md +53 -45
- package/docs/run-bundle.md +92 -0
- package/docs/runs-and-usage.md +17 -2
- package/docs/server.md +7 -3
- package/docs/sheets.md +9 -9
- package/docs/signal-channel.md +112 -0
- package/docs/speech.md +7 -1
- package/docs/sqlite-persistence.md +1 -1
- package/docs/supervisors.md +33 -5
- package/docs/telegram-channel.md +157 -0
- package/docs/testing.md +2 -2
- package/docs/thinking-and-reasoning.md +3 -1
- package/docs/tools.md +6 -5
- package/docs/web-tools.md +2 -1
- package/docs/wiki.md +1 -1
- package/docs/work-artifacts-and-review.md +14 -4
- package/docs/work-connectors.md +12 -10
- package/docs/work-sandbox.md +115 -0
- package/docs/work-tools.md +50 -18
- package/docs/workflows.md +69 -1
- package/docs/working-and-semantic-memory.md +25 -14
- package/package.json +5 -3
- package/templates/README.md +2 -0
- package/templates/business-worker/README.md.tmpl +19 -0
- package/templates/business-worker/env.example.tmpl +1 -0
- package/templates/business-worker/gitignore.tmpl +11 -0
- package/templates/business-worker/manifest.json +12 -0
- package/templates/business-worker/package.json.tmpl +23 -0
- package/templates/business-worker/src/agent.ts.tmpl +92 -0
- package/templates/business-worker/src/index.ts.tmpl +13 -0
- package/templates/business-worker/src/tests/agent.test.ts.tmpl +77 -0
- package/templates/business-worker/tsconfig.json.tmpl +15 -0
- package/templates/personal-assistant/README.md.tmpl +18 -0
- package/templates/personal-assistant/env.example.tmpl +1 -0
- package/templates/personal-assistant/gitignore.tmpl +11 -0
- package/templates/personal-assistant/manifest.json +11 -0
- package/templates/personal-assistant/package.json.tmpl +23 -0
- package/templates/personal-assistant/src/agent.ts.tmpl +65 -0
- package/templates/personal-assistant/src/index.ts.tmpl +13 -0
- package/templates/personal-assistant/src/tests/agent.test.ts.tmpl +28 -0
- package/templates/personal-assistant/tsconfig.json.tmpl +15 -0
|
@@ -4,10 +4,12 @@
|
|
|
4
4
|
|
|
5
5
|
`@arnilo/prism-memory/compaction/observational-memory` is an optional subpath for source-backed observational memory and fast compaction.
|
|
6
6
|
|
|
7
|
-
Current status: ledger/projection/render/recall utilities, explicit worker runtime, fast compaction strategy, inert extension helper, recall tool, and status/view command factories are available.
|
|
7
|
+
Current status: ledger/projection/render/recall utilities, optional work-scope indexing, explicit worker runtime, fast compaction strategy, inert extension helper, recall tool, and status/view command factories are available.
|
|
8
8
|
|
|
9
9
|
This package is distinct from `@arnilo/prism-memory` working/semantic memory: observational memory compresses and recalls source-backed observations/reflections; semantic memory retrieves embeddings; working memory stores the current structured profile/state. Hosts may compose both.
|
|
10
10
|
|
|
11
|
+
This page's memory stays **episodic**: the ledger records what happened in a session, its observer/reflector/dropper workers are the only writers of observations and reflections, and nothing downstream re-observes the transcript. Typed notes that want to outlive the session (facts, procedures, file references) are a separate layer — the [memory fabric](memory-fabric.md) — which views an observation by id as an `episode` note without copying it, never wraps or replaces these workers, and keeps its own recall path. Promotion out of the ledger is an explicit host write, not a side effect of compaction.
|
|
12
|
+
|
|
11
13
|
## Four-layer provider context
|
|
12
14
|
|
|
13
15
|
Observational memory composes four independent layers for long sessions (Mastra-style):
|
|
@@ -19,7 +21,9 @@ Observational memory composes four independent layers for long sessions (Mastra-
|
|
|
19
21
|
| **Reflections** | Higher-level summaries over observation ids | Reflector worker on observations after last reflection coverage when `reflection.observationTokens` met |
|
|
20
22
|
| **Raw-source retrieval** | Exact branch messages behind a memory id or cursor page | `recallObservationalMemory()` / `recallObservationalMemoryBranchPage()` / `createRecallMemoryTool()` — exact-id or cursor paging only; no semantic search |
|
|
21
23
|
|
|
22
|
-
|
|
24
|
+
The opt-in work-scope index filters the observation and reflection layers for a host-selected working set. It is not a fifth context layer, retrieval system, or session scope.
|
|
25
|
+
|
|
26
|
+
Activation is explicit: `createObservationalMemory().attach()` coordinates post-run observe/reflect/drop and compaction — by default `context.compactAfterTokens`, or whatever host gate `trigger` / `shouldCompact` supplies. Import and extension `setup` start nothing. Recall, commands, and utilities fail closed on invalid ids, wrong `sessionId`, ambiguous tool input, or oversized pages. Pass `secrets` for exact-value redaction in render/recall/worker paths. Branch isolation: hosts supply current-branch `appendEntry` and `getEntries`; mismatched store/session pairs fail closed after append.
|
|
23
27
|
|
|
24
28
|
See `examples/observational-memory-lifecycle.ts` for attach → turn → projection/recall/page without live credentials.
|
|
25
29
|
|
|
@@ -31,6 +35,8 @@ Use `createObservationalMemoryCompactionStrategy()` when compaction should rende
|
|
|
31
35
|
|
|
32
36
|
## Inputs / request
|
|
33
37
|
|
|
38
|
+
**Option surfaces** — `appendEntry` takes `ObservationalMemoryAppendOptions` (custom observation/reflection text, trust, and metadata); `createWorkScopeController` takes `WorkScopeControllerOptions` (session, `appendEntry`, optional `secrets`).
|
|
39
|
+
|
|
34
40
|
Memory records use `SessionEntry.kind: "custom"` with `entry.data.type` markers:
|
|
35
41
|
|
|
36
42
|
| Type | Payload |
|
|
@@ -38,6 +44,9 @@ Memory records use `SessionEntry.kind: "custom"` with `entry.data.type` markers:
|
|
|
38
44
|
| `om.observations.recorded` | `{ observations, coversUpToId? }` — successful observer runs append coverage even when `observations` is empty. |
|
|
39
45
|
| `om.reflections.recorded` | `{ reflections, coversUpToId? }` |
|
|
40
46
|
| `om.observations.dropped` | `{ observationIds, coversUpToId? }` |
|
|
47
|
+
| `om.scope.opened` | `{ id, parentId?, kind?, label? }` — host-defined scope tree node. |
|
|
48
|
+
| `om.scope.closed` / `om.scope.entered` / `om.scope.left` | `{ scopeId }` for close/enter; `{}` for leave. |
|
|
49
|
+
| `om.scope.bound` / `om.scope.unbound` | `{ scopeId, refs }` — many-to-many `om:<12-hex>` or `reflection:<12-hex>` membership. |
|
|
41
50
|
| `om.folded` | Compaction `data.memory` folded details. |
|
|
42
51
|
|
|
43
52
|
Ids are known, source-backed 12-character lowercase hex strings matching `^[a-f0-9]{12}$`.
|
|
@@ -58,7 +67,7 @@ Worker limits are finite positive safe integers:
|
|
|
58
67
|
| Recent-message window | — | 512 KiB | `renderRecentMessageWindow()` hard cap |
|
|
59
68
|
| Recall page size | 20 | 100 | `retrieval.pageLimit` / recall tool `limit` |
|
|
60
69
|
|
|
61
|
-
Direct `runObserver()` / `runReflector()` / `runDropper()` calls retain required `maxTurns` and accept the corresponding shorter worker fields (`maxToolCalls`, `maxResultBytes`, etc.). Named default/hard constants and `resolveMemoryWorkerLimits()` are exported.
|
|
70
|
+
Direct `runObserver()` / `runReflector()` / `runDropper()` calls retain required `maxTurns` and accept the corresponding shorter worker fields (`maxToolCalls`, `maxResultBytes`, etc.). Workers are tool-only: text, thinking, and done events are ignored; a turn with no `tool_call` succeeds as a no-op and records nothing. Named default/hard constants and `resolveMemoryWorkerLimits()` are exported.
|
|
62
71
|
|
|
63
72
|
## Outputs / response / events
|
|
64
73
|
|
|
@@ -67,18 +76,21 @@ Key exports:
|
|
|
67
76
|
| Export | Purpose |
|
|
68
77
|
| --- | --- |
|
|
69
78
|
| `foldObservationalMemoryLedger()` | Fold custom memory entries into observations, reflections, drops, and coverage markers. |
|
|
79
|
+
| `foldWorkScopeMap()` / `createWorkScopeController()` | Fold the opt-in scope index or append validated open/close/enter/leave/bind/unbind entries. |
|
|
80
|
+
| `projectWorkMemory()` | Filter the folded observations/reflections through a scope query; exact-id recall stays unfiltered. |
|
|
81
|
+
| `withWorkScope()` | Open a missing scope, enter it for an async callback, and always leave without closing it. |
|
|
70
82
|
| `isEligibleObservationSourceEntry()` / `eligibleObservationSources()` | Select user/assistant/tool `message` entries for observer input. |
|
|
71
83
|
| `unscannedEntries()` / `observationsUncoveredByReflection()` | Dual coverage helpers for observation scan and reflection windows. |
|
|
72
|
-
| `buildObservationalMemoryProjection()` | Build active/full/folded projections from current branch entries. |
|
|
73
|
-
| `buildObservationalMemoryContextBlocks()` | Render observational-memory + recent-messages context blocks for provider input. |
|
|
84
|
+
| `buildObservationalMemoryProjection()` | Build active/full/folded projections from current branch entries. Optional `invalidatedIds` drops observations whose id or `sourceEntryIds` match, and reflections that rest on them. Full ledger stays for audit. |
|
|
85
|
+
| `buildObservationalMemoryContextBlocks()` | Render observational-memory + recent-messages context blocks for provider input. Same `invalidatedIds` option. |
|
|
74
86
|
| `selectRecentMessageEntries()` / `renderRecentMessageWindow()` | Bounded exact recent-message suffix; count via `keepRecentEntries`, optional token trim via `estimateEntryTokens`. |
|
|
75
87
|
| `createFoldedMemoryDetails()` | Create JSON details for compaction `data.memory`. |
|
|
76
88
|
| `renderObservationalMemory()` | Render reflections and observations into a prepared memory summary. |
|
|
77
|
-
| `recallObservationalMemory()` | Recover source evidence for a known observation/reflection id from supplied current-branch entries. |
|
|
89
|
+
| `recallObservationalMemory()` | Recover source evidence for a known observation/reflection id from supplied current-branch entries. `invalidatedIds` withholds content (`reason: "revoked"`) without injecting derived text. |
|
|
78
90
|
| `recallObservationalMemoryBranchPage()` | Page eligible user/assistant/tool messages around a cursor entry id (`forward`/`backward`, optional `detail: summary|full`). |
|
|
79
91
|
| `createMemoryId()` / `isMemoryId()` | Create/check 12-character ids. |
|
|
80
92
|
| `resolveObservationalMemorySettings()` | Merge `observational-memory` settings with defaults and overrides. |
|
|
81
|
-
| `createObservationalMemory()` / `attach()` | One activation wires post-run observe/reflect/drop and `compactAfterTokens`
|
|
93
|
+
| `createObservationalMemory()` / `attach()` | One activation wires post-run observe/reflect/drop and compaction (`compactAfterTokens`, or a host `trigger` / `shouldCompact`); returns proxied session, runtime, context provider, and strategy. |
|
|
82
94
|
| `createObservationalMemoryRuntime()` | Low-level explicit flush for advanced hosts or tests. |
|
|
83
95
|
| `createObservationalMemoryCompactionStrategy()` | Render existing folded memory as a standard Prism compaction summary with `data.memory`. |
|
|
84
96
|
| `createObservationalMemoryExtension()` | Inert extension helper that registers the strategy contribution unless disabled. |
|
|
@@ -86,7 +98,32 @@ Key exports:
|
|
|
86
98
|
| `createMemoryStatusCommand()` / `createMemoryViewCommand()` | Optional `om:status` and `om:view` command factories. |
|
|
87
99
|
| `createObservationalMemoryCommands()` | Convenience factory returning status and view commands. |
|
|
88
100
|
|
|
89
|
-
Pure utilities create no events, workers, tools, commands, credentials, or provider requests. `createObservationalMemoryExtension()` and import alone start nothing. `createObservationalMemory().attach()` runs workers only after proxied `run`/`prompt`/`stream`/`compact` complete (or after `wrapResumeRun` / `wrapResumeStream`). `createObservationalMemoryRuntime().flush()` remains for manual/advanced use. Attached `contextProvider` renders two blocks each turn: `observational-memory` (active reflections/observations aligned to the recent-message boundary) and `recent-messages` (last `keepRecentEntries` message entries in branch order, optionally trimmed by `recentMessageMaxTokens` using `estimateEntryTokens`; oldest dropped first). Compaction uses the same `keepRecentEntries` setting. Observer input includes only eligible `message` entries (`user`, `assistant`, `tool`); memory/compaction/bookkeeping entries advance `coversUpToId` scan coverage without entering the observer prompt. Successful observer/reflector runs append coverage markers even when they record zero facts. Reflection uses only active observations recorded after the last `om.reflections.recorded` entry unless `flush({ fullReflectionRebuild: true })`. Attached `flush()` skips with `run_active` while a proxied run is in flight. The compaction strategy is O(n) over supplied entries and makes no provider call.
|
|
101
|
+
Pure utilities create no events, workers, tools, commands, credentials, or provider requests. `createObservationalMemoryExtension()` and import alone start nothing. `createObservationalMemory().attach()` runs workers only after proxied `run`/`prompt`/`stream`/`compact` complete (or after `wrapResumeRun` / `wrapResumeStream`). `createObservationalMemoryRuntime().flush()` remains for manual/advanced use. Attached `contextProvider` renders two blocks each turn: `observational-memory` (active reflections/observations aligned to the recent-message boundary) and `recent-messages` (last `keepRecentEntries` message entries in branch order, optionally trimmed by `recentMessageMaxTokens` using `estimateEntryTokens`; oldest dropped first). Compaction uses the same `keepRecentEntries` setting. Observer input includes only eligible `message` entries (`user`, `assistant`, `tool`); memory/compaction/bookkeeping entries advance `coversUpToId` scan coverage without entering the observer prompt. Successful observer/reflector runs append coverage markers even when they record zero facts. Reflection uses only active observations recorded after the last `om.reflections.recorded` entry unless `flush({ fullReflectionRebuild: true })`. Attached `flush()` skips with `run_active` while a proxied run is in flight. The compaction strategy is O(n) over supplied entries and makes no provider call.
|
|
102
|
+
|
|
103
|
+
### Work-scope index (opt-in)
|
|
104
|
+
|
|
105
|
+
`WorkScope` is a host-named, append-only index over one observational-memory ledger. Without `om.scope.*` entries, the map has only its implicit `session` root, context renders the existing active pool, and the dropper keeps its existing behavior.
|
|
106
|
+
|
|
107
|
+
Use `createWorkScopeController({ session, appendEntry, secrets? })` to `open`, `close`, `enter`, `leave`, `bind`, or `unbind` scopes. Scope ids are host-defined (`[A-Za-z0-9._:/-]{1,128}`, no `..`); there are caps of 256 scopes, depth/stack 8, 4,096 binds per scope, and 512 characters for labels or kinds. Invalid ids, missing/closed parents, duplicate scopes, unknown record ids, and ownership mismatch fail closed. Labels and kinds receive the same secret redaction as observational-memory text.
|
|
108
|
+
|
|
109
|
+
`projectWorkMemory(ledger, map, { from, include, closed?, kinds? })` returns a filtered observation/reflection view plus outline. `include` is `self`, `self+ancestors`, `self+descendants`, or `lineage`; `closed: "hide"` is the default, except closed ancestors of `from` remain available. Default attached context uses the current leaf with `self+ancestors`, rendering Scope Outline, Reflections, then Observations. The compaction summary — the layer the next run's pack starts from — renders the same projection, so the full ledger never rides into the prefix; the folded payload keeps every observation, so entering another scope can still surface what that summary hid. `recallObservationalMemory()` still reads the complete current branch by exact id.
|
|
110
|
+
|
|
111
|
+
After a flush records new observations or reflections, it binds those ids once to the current leaf scope only. A host promotes relevant memory explicitly by binding it to an ancestor; a reflection whose bind sits on a **closed** scope can also graduate into durable semantic memory through the fabric's `remember({ kind: "fact" | "procedure", reflectionId })`. While any host scope exists, the runtime skips the observation dropper; the folded-payload byte cap remains a storage safety cap, not working-set garbage collection. `withWorkScope(controller, spec, fn)` opens `spec` if needed, enters it, runs `fn`, and leaves in `finally`; it never closes a scope. This index does not provide resource-scoped observational memory or budget-based dropping as a working-set mechanism.
|
|
112
|
+
|
|
113
|
+
### Compact-when override
|
|
114
|
+
|
|
115
|
+
`createObservationalMemory()` accepts a compact-when gate beside the settings: `trigger` (the same union `CompactionOptions.trigger` uses) or the `shouldCompact(context)` shorthand. When either is set it **replaces** `context.compactAfterTokens`; omitted, the token gate is unchanged.
|
|
116
|
+
|
|
117
|
+
```ts
|
|
118
|
+
const om = createObservationalMemory({
|
|
119
|
+
observation: { provider, model },
|
|
120
|
+
shouldCompact: (context) => context.entryCount > 40 || context.estimatedInputTokens / context.inputCapTokens >= 0.9,
|
|
121
|
+
});
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
- `context.entryCount` counts current-branch entries; `context.estimatedInputTokens` is this package's own `estimateEntryTokens` sum; `context.inputCapTokens` comes from `resolveInputCap` on `attach({ sessionModel })`.
|
|
125
|
+
- Both token numbers resolve lazily, so a callback that only reads counts works with a `sessionModel` that declares no `contextWindow`. Reading the cap without one throws inside the callback, and the gate then decides **false** — the sync loop reports it through the `debug` sink (`observational-memory:compaction-trigger-error`) and never compacts on a guess. `input_ratio` needs a resolvable cap and throws instead.
|
|
126
|
+
- An unknown trigger `type` or a non-function `shouldCompact` throws at `createObservationalMemory()`, before any session work. Tool and command factories are inert until a host registers/selects them.
|
|
90
127
|
|
|
91
128
|
## Request/response example
|
|
92
129
|
|
|
@@ -105,8 +142,13 @@ import {
|
|
|
105
142
|
createObservationalMemoryCommands,
|
|
106
143
|
createObservationalMemoryRuntime,
|
|
107
144
|
createRecallMemoryTool,
|
|
145
|
+
createWorkScopeController,
|
|
146
|
+
foldObservationalMemoryLedger,
|
|
147
|
+
foldWorkScopeMap,
|
|
148
|
+
projectWorkMemory,
|
|
108
149
|
recallObservationalMemory,
|
|
109
150
|
renderObservationalMemory,
|
|
151
|
+
withWorkScope,
|
|
110
152
|
} from "@arnilo/prism-memory/compaction/observational-memory";
|
|
111
153
|
|
|
112
154
|
const om = createObservationalMemory({
|
|
@@ -119,11 +161,19 @@ const attached = om.attach(session, {
|
|
|
119
161
|
appendEntry: (entry, options) => store.append(entry, options),
|
|
120
162
|
sessionModel: agent.config.model,
|
|
121
163
|
});
|
|
122
|
-
|
|
164
|
+
const scopes = createWorkScopeController({ session: attached.session, appendEntry: (entry, options) => store.append(entry, options) });
|
|
165
|
+
await scopes.open({ id: "plan:memory", kind: "plan", label: "Memory work" });
|
|
166
|
+
await withWorkScope(scopes, { id: "task:cleanup", parentId: "plan:memory", kind: "task" }, () =>
|
|
167
|
+
attached.session.run("Continue from prior work"),
|
|
168
|
+
);
|
|
123
169
|
|
|
124
170
|
const entries = await session.entries();
|
|
125
171
|
const projection = buildObservationalMemoryProjection(entries);
|
|
126
|
-
const
|
|
172
|
+
const scoped = projectWorkMemory(foldObservationalMemoryLedger(entries), foldWorkScopeMap(entries), {
|
|
173
|
+
from: "task:cleanup",
|
|
174
|
+
include: "self+ancestors",
|
|
175
|
+
});
|
|
176
|
+
const summary = renderObservationalMemory(scoped.reflections, scoped.observations, { outline: scoped.outline });
|
|
127
177
|
const evidence = recallObservationalMemory(entries, "aaaaaaaaaaaa");
|
|
128
178
|
|
|
129
179
|
const memory = createObservationalMemoryRuntime({
|
|
@@ -231,11 +281,14 @@ Ownership: funnel only within the `OwnershipScope` already on the parent agent/s
|
|
|
231
281
|
## Related APIs
|
|
232
282
|
|
|
233
283
|
- [Use-case model selection](use-case-model-selection.md): session vs worker model binding and `resolveUseCaseModel`.
|
|
284
|
+
- [Attention compiler](attention-compiler.md): opt-in per-turn shrink that runs before compaction is considered and resolves the same input cap for `input_ratio` triggers.
|
|
234
285
|
- [Thinking and reasoning](thinking-and-reasoning.md): `thinkingLevel` → provider `compat`.
|
|
235
286
|
- [Provider request policies](provider-request-policies.md): derived `om:{session.id}` on worker generate.
|
|
236
287
|
- [Compaction and retry policies](compaction-and-retry.md): replaceable compaction strategy boundary.
|
|
288
|
+
- [Workflows](workflows.md): a host may use a workflow `nodeId` as a scope id with `withWorkScope`; the workflow runner does not enter scopes itself.
|
|
237
289
|
- [LLM compaction package](compaction-llm.md): existing optional compaction-package pattern.
|
|
238
290
|
- [Session stores and branching](session-stores-and-branching.md): branch entries that observational memory reads and appends to.
|
|
291
|
+
- [Memory fabric](memory-fabric.md): optional typed notes over the same stores; `episode` notes are views of these observation ids, and the ledger stays episodic.
|
|
239
292
|
- [Supervisor delegation](supervisors.md): child sessions whose messages this page's opt-in funnel can copy onto a workspace branch.
|
|
240
293
|
- [Extensions](extensions.md): inert registration pattern for optional package contributions.
|
|
241
294
|
- [Tools](tools.md): host activation and dispatch for optional recall tool contributions.
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
# Connected apps
|
|
2
|
+
|
|
3
|
+
## What it does
|
|
4
|
+
|
|
5
|
+
`createConnectedAppSession()` groups host-selected MCP bridges under one verified identity. It exposes only prefixed `ToolDefinition`s selected by the host and leaves transport construction, OAuth, credentials, and remote effect classification with that host.
|
|
6
|
+
|
|
7
|
+
## When to use it
|
|
8
|
+
|
|
9
|
+
Use connected apps when one agent needs a small, identity-scoped set of SaaS or internal MCP servers. Use typed [work tools](work-tools.md) instead for high-trust Google Workspace or Microsoft 365 actions that require Prism's draft/approve lifecycle.
|
|
10
|
+
|
|
11
|
+
## Inputs / request
|
|
12
|
+
|
|
13
|
+
| Input | Required | Contract |
|
|
14
|
+
| --- | --- | --- |
|
|
15
|
+
| `identity` | yes | Active, verified `AgentIdentity` bound to every connection. A `bind()` identity, if supplied, must match tenant, account, user, and principal. |
|
|
16
|
+
| `select` | yes | Host admission callback. `false` denies before any MCP connection. |
|
|
17
|
+
| `effect` | no | Shared `McpToolEffectPolicy`. Omit it to retain MCP's `external_mutation` / `unsupported` default. |
|
|
18
|
+
| `connect` | no | Test seam. Production uses `connectMcpTools`. |
|
|
19
|
+
| `maxApps` | no | Maximum bindings; defaults to 8 and has a hard cap of 32. |
|
|
20
|
+
| `bind({ appId, serverId, transport, allowTools, effect })` | yes | `appId` and `serverId` are unique session identifiers. `transport` is already host-built. `allowTools` is an exact remote-name allowlist; per-binding `effect` overrides the shared policy. |
|
|
21
|
+
|
|
22
|
+
## Outputs / response / events
|
|
23
|
+
|
|
24
|
+
`bind()` connects one admitted bridge. `tools()` returns its prefixed tools without re-listing. `refresh()` re-lists every bound bridge. `list()` returns only `appId`, `serverId`, and visible prefixed tool names. `unbind()` and `close()` close their bridges.
|
|
25
|
+
|
|
26
|
+
## Host composition inspection
|
|
27
|
+
|
|
28
|
+
Pass only `apps.list()` identifiers into `inspectHostComposition()`; inspection never connects, refreshes, or receives a transport. A business host with connected apps must provide a verified identity.
|
|
29
|
+
|
|
30
|
+
```ts
|
|
31
|
+
const bindings = apps.list();
|
|
32
|
+
const report = inspectHostComposition({
|
|
33
|
+
profile: "business",
|
|
34
|
+
agent,
|
|
35
|
+
connectedApps: {
|
|
36
|
+
appIds: bindings.map(({ appId }) => appId),
|
|
37
|
+
serverIds: bindings.map(({ serverId }) => serverId),
|
|
38
|
+
},
|
|
39
|
+
});
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## Request/response example
|
|
43
|
+
|
|
44
|
+
```json
|
|
45
|
+
{
|
|
46
|
+
"binding": {
|
|
47
|
+
"appId": "slack",
|
|
48
|
+
"serverId": "slack",
|
|
49
|
+
"transport": { "type": "stdio", "command": "/usr/bin/slack-mcp", "args": ["mcp"] },
|
|
50
|
+
"allowTools": ["list_channels", "post_message"]
|
|
51
|
+
},
|
|
52
|
+
"list": [{ "appId": "slack", "serverId": "slack", "tools": ["mcp:slack:list_channels", "mcp:slack:post_message"] }]
|
|
53
|
+
}
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## Implementation example
|
|
57
|
+
|
|
58
|
+
```ts
|
|
59
|
+
import { createToolRegistry, type AgentIdentity } from "@arnilo/prism";
|
|
60
|
+
import { createConnectedAppSession } from "@arnilo/prism-mcp";
|
|
61
|
+
|
|
62
|
+
const identity: AgentIdentity = {
|
|
63
|
+
tenantId: "tenant-a",
|
|
64
|
+
userId: "user-a",
|
|
65
|
+
principal: { kind: "user", id: "user-a" },
|
|
66
|
+
scopes: ["tools:execute"],
|
|
67
|
+
issuedAt: new Date().toISOString(),
|
|
68
|
+
verified: true,
|
|
69
|
+
};
|
|
70
|
+
const apps = createConnectedAppSession({
|
|
71
|
+
identity,
|
|
72
|
+
select: ({ transport }) => transport.type === "stdio" && transport.command === "/usr/bin/slack-mcp",
|
|
73
|
+
effect: ({ remoteName }) =>
|
|
74
|
+
remoteName.startsWith("list_") ? { kind: "none", idempotency: "none" } : undefined,
|
|
75
|
+
});
|
|
76
|
+
await apps.bind({
|
|
77
|
+
appId: "slack",
|
|
78
|
+
serverId: "slack",
|
|
79
|
+
transport: { type: "stdio", command: "/usr/bin/slack-mcp", args: ["mcp"] },
|
|
80
|
+
allowTools: ["list_channels", "post_message"],
|
|
81
|
+
});
|
|
82
|
+
|
|
83
|
+
const registry = createToolRegistry({ duplicate: "error" });
|
|
84
|
+
for (const tool of apps.tools()) registry.register(tool);
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
## Slack MCP wrap example
|
|
88
|
+
|
|
89
|
+
[`examples/connected-slack-mcp.ts`](../examples/connected-slack-mcp.ts) is a network-free template: a mock bridge exposes read and write Slack names, an exact `allowTools` list exposes only the intended tools, and the host policy marks `list_*`, `get_*`, and `search_*` as observations. `post_*`, `update_*`, and `delete_*` remain the MCP external-mutation default.
|
|
90
|
+
|
|
91
|
+
For a real Slack server, the host `select` callback must admit its exact stdio command or HTTP origin. Build credentials into stdio `env` or MCP OAuth before `bind()`; never place tokens in model context.
|
|
92
|
+
|
|
93
|
+
## Open Connector sidecar (example only)
|
|
94
|
+
|
|
95
|
+
[`examples/open-connector-sidecar/`](../examples/open-connector-sidecar/README.md) shows the opposite shape: Prism as the host for a sibling connector gateway over loopback MCP. The recipe pins an immutable `ghcr.io/oomol-lab/open-connector` release tag (never `main`/`tip`/`latest`), binds port `3000` on `127.0.0.1`, and keeps runtime tokens host-supplied.
|
|
96
|
+
|
|
97
|
+
Admission stays with the host: `select` must admit the exact loopback origin with `allowLoopbackHttp: true`, and an exact `allowTools` list (`search_actions`, `get_action_guide`, `execute_action`) keeps the provider catalog and connection listings off the model. Reads stay observations; `execute_action` is intentionally left unclassified and therefore remains `external_mutation` / `unsupported`. MCP `execute_action` accepts no `Idempotency-Key` — use HTTP `POST /v1/actions/:actionId` for retry-safe writes, or keep writes in [work tools](work-tools.md).
|
|
98
|
+
|
|
99
|
+
Identity mapping is host glue: Open Connector has no Prism identity, so the host resolves `connectionName`/`x-oo-connector-alias` from the verified `AgentIdentity` and issues one runtime token per identity. OC provider egress (including `skipDnsValidation` executors and `OOMOL_CONNECT_ALLOWED_PROXIES`) stays inside Open Connector; Prism's `pinnedFetch` policy cannot be layered over it.
|
|
100
|
+
|
|
101
|
+
No workspace package depends on Open Connector, no OC source is vendored, and `examples/open-connector-sidecar.ts` proves the admission and effect behavior network-free.
|
|
102
|
+
|
|
103
|
+
## Extension and configuration notes
|
|
104
|
+
|
|
105
|
+
Build stdio `env` and Streamable HTTP `requestInit.headers` in host code before `bind()`. Use `createMcpOAuthTransport()` when the host chooses MCP OAuth. The session does not discover catalogs, construct commands, resolve credentials, or add a second OAuth implementation.
|
|
106
|
+
|
|
107
|
+
## Security and performance notes
|
|
108
|
+
|
|
109
|
+
`select` is mandatory and deny-by-default. The session never infers an effect from remote descriptions or annotations; unclassified tools stay external mutations with unsupported idempotency. `allowTools` is an allowlist. `list()` excludes transports, headers, environment, and tokens. Binding performs one MCP connect; `tools()` uses cached bridge definitions until explicit `refresh()`.
|
|
110
|
+
|
|
111
|
+
## Related APIs
|
|
112
|
+
|
|
113
|
+
- [MCP client bridge and server exposure](mcp-tools.md): underlying MCP transports, bridge limits, OAuth, and tool mapping.
|
|
114
|
+
- [Agent identity](agent-identity.md): verified identity lifecycle and delegation boundaries.
|
|
115
|
+
- [Recoverable tool effects](tool-effects.md): effect declarations and mutation recovery semantics.
|
|
116
|
+
- [Work tools](work-tools.md): typed high-trust M365 and Google Workspace actions.
|
|
@@ -128,7 +128,7 @@ Each active skill contributes two things the runtime wires together:
|
|
|
128
128
|
- `Skill` prompt text → rendered as system messages by `skillMessages()` / `skillPromptText()` (active set only). Default `skillsDisclosure: "progressive"` sends `Skill <name>: <description>`; full `instructions` appear only when the skill is in the session `LoadedSkillSet` or disclosure is `"eager"`.
|
|
129
129
|
- `Skill.context: ContextProvider[]` → collected across active skills (`activeSkills.flatMap(s => s.context ?? [])`), resolved through the existing `resolveContextProviders(...)`, and merged into the request's `context` **after** host `AgentConfig.context` blocks. Inactive skills contribute neither instructions nor context.
|
|
130
130
|
|
|
131
|
-
`toolNames` enforcement is live: because selection routes through `resolveActiveSkills()`, a skill demanding a host-inactive tool throws with `Skill ${name} requires inactive tool: ${missing}` **before the first provider turn** — no provider call, no store write, no partial side effect.
|
|
131
|
+
`toolNames` enforcement is live: because selection routes through `resolveActiveSkills()`, a skill demanding a host-inactive tool throws with `Skill ${name} requires inactive tool: ${missing}` **before the first provider turn** — no provider call, no store write, no partial side effect. When `RunOptions.toolNames` narrows the run, that snapshot is the host-active list — a skill cannot require a registered tool the run did not grant.
|
|
132
132
|
|
|
133
133
|
```ts
|
|
134
134
|
import { createAgent, createSkillRegistry, type ContextProvider } from "@arnilo/prism";
|
|
@@ -211,6 +211,19 @@ Mode slices and skill bodies are independent: the injector can add `PONYTAIL MOD
|
|
|
211
211
|
|
|
212
212
|
Pure validation without the tool: `resolveSkillLoad({ registry, name, tools, loaded, activeSkillNames })`.
|
|
213
213
|
|
|
214
|
+
### Bundled work skills (`docx`, `xlsx`, `powerpoint`, `pdf`)
|
|
215
|
+
|
|
216
|
+
`@arnilo/prism-work/skills` vendors four MIT Hermes productivity skills. `loadWorkSkills()` reads the committed `SKILL.md` files (64 KiB cap) and overlays Prism `toolNames` in TypeScript — vendored markdown is never edited.
|
|
217
|
+
|
|
218
|
+
```ts
|
|
219
|
+
import { createSkillRegistry } from "@arnilo/prism";
|
|
220
|
+
import { loadWorkSkills } from "@arnilo/prism-work/skills";
|
|
221
|
+
|
|
222
|
+
const registry = createSkillRegistry(loadWorkSkills(), { duplicate: "error" });
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
Scripts run only via work-sandbox `execFile` (argv, no shell). See [Work sandbox](work-sandbox.md).
|
|
226
|
+
|
|
214
227
|
### Context budget priority and skill demotion
|
|
215
228
|
|
|
216
229
|
When `assembleProviderInput` runs with `contextBudget`, `applyContextBudget` evicts droppable sections in layout order. Within `context` blocks and skills, victims sort by ascending `ContextBlock.priority` (missing = **0**), then LIFO within the same priority.
|
|
@@ -219,7 +232,7 @@ Under pressure on a skill with a loaded body, eviction may demote to catalog-onl
|
|
|
219
232
|
|
|
220
233
|
### Optional tool-result fold
|
|
221
234
|
|
|
222
|
-
`toolResultFold` on `AgentConfig` / `RunOptions` (run wins) is **off** unless the host supplies a `summarize` callback. When enabled, aged large tool-result messages in the **provider view** become a one-line header plus bounded summary text; session store entries stay raw. Defaults: `minAgeTurns` **2**, `minBytes` **4096**, `maxSummaryBytes` **512** (hard **4096**). Summarizer failure keeps the raw tool result (fail closed). Not a second memory system — use observational memory / compaction for durable recall.
|
|
235
|
+
`toolResultFold` on `AgentConfig` / `RunOptions` (run wins) is **off** unless the host supplies a `summarize` callback. When enabled, aged large tool-result messages in the **provider view** become a one-line header plus bounded summary text; session store entries stay raw. Defaults: `minAgeTurns` **2**, `minBytes` **4096**, `maxSummaryBytes` **512** (hard **4096**). Summarizer failure keeps the raw tool result (fail closed). Not a second memory system — use observational memory / compaction for durable recall, and remember that observational memory stays the source-backed **episodic** ledger of the session: typed, time-bounded notes that outlive a session are a separate layer ([memory fabric](memory-fabric.md)) whose provider arrives through this same inert seam.
|
|
223
236
|
|
|
224
237
|
```ts
|
|
225
238
|
await session.run("…", {
|
|
@@ -267,8 +280,10 @@ Use `activateAllCapabilities: true` only as a temporary all-skills/all-tools com
|
|
|
267
280
|
|
|
268
281
|
- [Agent/session runtime](agent-session-runtime.md): consumes host-selected context providers and skills from explicit agent config.
|
|
269
282
|
- [Input and prompt assembly](input-and-prompt-assembly.md): default prompt builder and provider-input assembly helper.
|
|
283
|
+
- [Attention compiler](attention-compiler.md): opt-in ratio gate that rewrites aged history and tool results for one request, leaving resolved context blocks and skills in place.
|
|
270
284
|
- [Instruction injection](instruction-injection.md): package injectors contribute `contextBlocks` that merge after host+skill provider blocks.
|
|
271
285
|
- [Retrieval-augmented generation](rag.md): optional retrieved citations contribute through the same explicit inert context seam.
|
|
286
|
+
- [Memory fabric](memory-fabric.md): optional typed notes whose provider contributes the same `working-memory` / `semantic-memory` blocks this seam already carries, under a host-registered name.
|
|
272
287
|
- [Public contracts](public-contracts.md): `ContextProvider`, `ContextResolutionContext`, `ContextBlock`, `Skill`, `SkillRegistry`, `PromptBuilder`, and `PromptBuildRequest`.
|
|
273
288
|
- [Middleware hooks](middleware-hooks.md): `context` and `prompt_build` hooks.
|
|
274
289
|
- [Contribution registries](contribution-registries.md): inert context provider and skill contributions.
|
package/docs/conversations.md
CHANGED
|
@@ -117,7 +117,7 @@ Behavior notes:
|
|
|
117
117
|
- `create` with an explicit `id` writes with `expectedVersion: 0` (create-only): a duplicate create never overwrites the winner's metadata and returns the existing thread.
|
|
118
118
|
- `branch()` enforces `maxActiveBranches` on its read snapshot; the version guard inside the same write makes the cap exact even under concurrency (a concurrent branch cannot slip past the cap), and the marker keeps branch refs append-only.
|
|
119
119
|
- `archive()` on an already-archived thread is a no-op; a stale `branch`/`archive` racing a delete fails `not_found` (the row is gone) and the write path never re-creates a deleted thread — delete wins.
|
|
120
|
-
- Deletion purges the whole session ledger (entries, runs, events, tool calls, usage, branches, search rows) through `lifecycle.applyRetention`; legal holds block deletion and report `held: true`.
|
|
120
|
+
- Deletion purges the whole session ledger (entries, runs, events, tool calls, usage, branches, search rows) through `lifecycle.applyRetention`; legal holds block deletion and report `held: true`. Semantic-memory `forget({ hold: true })` is a separate knowledge-store hold (see [Working and semantic memory](working-and-semantic-memory.md)); conversation delete does not walk embeddings.
|
|
121
121
|
|
|
122
122
|
## Security and performance notes
|
|
123
123
|
|
package/docs/core.md
CHANGED
|
@@ -41,7 +41,6 @@ Every peer below is optional and fails closed at first use; the [optional peer d
|
|
|
41
41
|
| `@arnilo/prism-core/governance/observability` | OpenTelemetry instrumentation and event tracing | `@opentelemetry/api` |
|
|
42
42
|
| `@arnilo/prism-core/credentials/node` | Keyring-backed encrypted credential store, scrypt envelope encryption, OAuth2 PKCE providers, and OIDC identity verification | `@napi-rs/keyring` (bundled) |
|
|
43
43
|
| `@arnilo/prism-core/enterprise/postgres` | Unified multi-tenant enterprise PostgreSQL state (approvals, evaluations, model-router, policy, tool effects, work idempotency) | `pg` |
|
|
44
|
-
| `@arnilo/prism-core/integrations/work` | Microsoft 365 and Google Workspace CLI tool adapters with approval gates and idempotency | — |
|
|
45
44
|
| `@arnilo/prism-core/validation/json-schema` | Ajv-backed JSON Schema tool argument validation | `ajv` (bundled) |
|
|
46
45
|
|
|
47
46
|
## Usage Examples
|
|
@@ -85,3 +84,4 @@ const validator = createJsonSchemaToolArgumentValidator();
|
|
|
85
84
|
- Subpaths never load database drivers (`pg`, `better-sqlite3`) unless the specific database subpath is imported.
|
|
86
85
|
- All database and network drivers fail closed with clear actionable error messages when peers are omitted.
|
|
87
86
|
- Root `@arnilo/prism` remains dependency-free contracts and CLI runner.
|
|
87
|
+
- Messaging channels are `@arnilo/prism-channels` (`/telegram`, `/signal`), not a `@arnilo/prism-core` subpath.
|
package/docs/dev-inspector.md
CHANGED
|
@@ -57,6 +57,9 @@ Data-defined route table over the server seam — each route either rewrites the
|
|
|
57
57
|
| `GET /events?runId=<id>` | Durable SSE stream of normalized events. `Last-Event-ID` header reconnect and `?cursor=` are honored by the server seam; missing `runId` → `400 ERR_PRISM_DEV_ROUTE`. |
|
|
58
58
|
| `GET /runs/:id/replay?cursor=…` | Paged replay of a stored run from the durable `AgentEventSource` — **no session, no provider, no re-execution** (`createPrismAgentEventReplay` page). Returns `{ items, nextCursor?, terminal }`; unknown/foreign run ids → `404`. |
|
|
59
59
|
| `POST /runs/:runId/decisions/:decisionId` | Resumes/denies one suspended approval. Body `{ outcome: "allow_once" \| "allow_always" \| "deny", expectedVersion? }` → forwarded as a single-entry core decision batch; unknown discriminants and stale versions fail closed (`400`) at the core boundary **before any state write**. |
|
|
60
|
+
| `GET /inspect` | Returns the host composition report (`HostCompositionReport`) detailing profile, effective tools, redacted credentials, ownership, storage durability, sandbox capabilities, and governance coverage. | dev composition inspection (`inspectDevInspector`). |
|
|
61
|
+
| `GET /runs/:id/summary` | Projects durable replay events with `projectAgentTimeline({ content: "metadata" })` and returns `summarizeTimeline` (latency, cost, tool counts). No step I/O. Requires `eventSource`. Unknown run → `404`. |
|
|
62
|
+
| `POST /compare` | Body `{ left, right }` each `{ summary, aggregate?, manifest? }` — existing TimelineSummary / ExperimentAggregate fields only. Returns quality/cost/latency winners. `invariantsPassed: false` on either side sets `qualityWinner: "invariant_blocked"`. Bounded 64KiB. |
|
|
60
63
|
|
|
61
64
|
Reconnect semantics: every SSE frame carries `id: <cursor>`; a reconnecting client sends `Last-Event-ID: <cursor>` and receives exactly the post-cursor events — no duplicates, no loss (server conformance-tested). Replay pages are bounded by the deployment limits (`maxReplayEvents`, `maxReplayCursorBytes`) and ownership-scoped by the source seam itself.
|
|
62
65
|
|
|
@@ -71,6 +74,7 @@ Panels:
|
|
|
71
74
|
- **Usage** — per-run totals summed from `provider_turn_finished.usage` and the terminal `agent_finished.usage` (input/output/total tokens, cost when the model reports it).
|
|
72
75
|
- **Decisions** — `agent_suspended` renders one card per pending decision (`PendingDecision.approvalId`, tool name, redacted reason, `expectedVersion` from the event's run version). Buttons post `POST /runs/:runId/decisions/:approvalId` ({ outcome: `allow_once` | `allow_always` | `deny`, expectedVersion }); rejections show the seam's fail-closed error verbatim, and a remaining-multi-decision suspension re-renders from the response's `runState.interruption`.
|
|
73
76
|
- **Run selector** — session runs (live + loaded) with status; a durable view of any past run loads via `GET {basePath}/events?runId=…` over `EventSource` — the seam's own `Last-Event-ID` reconnect applies. Without a durable event source wired, loading by runId surfaces that fact instead of pretending to replay.
|
|
77
|
+
- **Compare** — `Compare last 2` loads `/runs/:id/summary` for the two most recent runs and `POST /compare`. Renders quality/cost/latency winners from those artifacts. A high mean score cannot beat a failed invariant. Without durable events the panel says so instead of inventing numbers.
|
|
74
78
|
|
|
75
79
|
## Request/response example
|
|
76
80
|
|
package/docs/device-adapters.md
CHANGED
|
@@ -91,6 +91,7 @@ if (chunk.accepted) emit(redactDeviceTelemetry(createSecretRedactor([token]), fr
|
|
|
91
91
|
## Related APIs
|
|
92
92
|
|
|
93
93
|
- [Browser automation](browser-automation.md): verified-state checkpoints + reload/verify-before-side-effect for browser composition.
|
|
94
|
+
- [Realtime voice](realtime-voice.md): governed OpenAI Realtime orchestration on this admission contract.
|
|
94
95
|
- [Linux desktop control](computer-use-linux.md): first-party host-owned `computer-use-linux` MCP wrapper using this contract.
|
|
95
96
|
- [Conversations](conversations.md): durable threads that own the runs device sessions bind to.
|
|
96
97
|
- [Host security](host-security.md): approval, sandbox, and egress trust boundaries device adapters compose over.
|
package/docs/diagrams.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# Diagramming, draw.io embed client, and mxGraph XML validation (`@arnilo/prism-
|
|
1
|
+
# Diagramming, draw.io embed client, and mxGraph XML validation (`@arnilo/prism-work/diagrams`)
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
The `@arnilo/prism-
|
|
5
|
+
The `@arnilo/prism-work/diagrams` package provides an origin-enforced draw.io / diagrams.net iframe embed client, XXE-safe mxGraph XML validation, and byte-stable deterministic XML canonicalization for content hashing and visual artifact workflows in Prism applications and agent runtimes.
|
|
6
6
|
|
|
7
7
|
### Core Capabilities
|
|
8
8
|
|
|
@@ -17,7 +17,7 @@ The `@arnilo/prism-office/diagrams` package provides an origin-enforced draw.io
|
|
|
17
17
|
|
|
18
18
|
## When to use it
|
|
19
19
|
|
|
20
|
-
Use `@arnilo/prism-
|
|
20
|
+
Use `@arnilo/prism-work/diagrams` when applications, host workspaces, or autonomous agents need to:
|
|
21
21
|
1. Embed an interactive, self-hosted draw.io / diagrams.net editor inside a web or Electron iframe with strictly enforced cross-origin security.
|
|
22
22
|
2. Coordinate diagram editing lifecycles (`init` handshake, `load`, `save`, `autosave`, `merge`, and `export`).
|
|
23
23
|
3. Execute save-with-preview workflows generating SVG (`xmlsvg`) or PNG (`xmlpng`) visual snapshots from the active editor session.
|
|
@@ -148,7 +148,7 @@ Outbound host-to-editor action postMessage:
|
|
|
148
148
|
## Implementation example
|
|
149
149
|
|
|
150
150
|
```ts
|
|
151
|
-
import { createDrawioEmbed, validateDrawioXml, canonicalizeDrawioXml } from "@arnilo/prism-
|
|
151
|
+
import { createDrawioEmbed, validateDrawioXml, canonicalizeDrawioXml } from "@arnilo/prism-work/diagrams";
|
|
152
152
|
|
|
153
153
|
// 1. Initialize embed client with strict origin binding
|
|
154
154
|
const embed = createDrawioEmbed({
|
|
@@ -240,8 +240,8 @@ const summary = validateDrawioXml(xml, {
|
|
|
240
240
|
|
|
241
241
|
## Related APIs
|
|
242
242
|
|
|
243
|
-
- [`@arnilo/prism-
|
|
244
|
-
- [`@arnilo/prism-
|
|
243
|
+
- [`@arnilo/prism-work/documents`](./documents.md): Specification-compliant OpenXML document generation and preview rendering for DOCX, XLSX, and PPTX.
|
|
244
|
+
- [`@arnilo/prism-work/sheets`](./sheets.md): Spreadsheet and CSV parsing engine with strict financial decimal safety guarantees.
|
|
245
245
|
- [`@arnilo/prism-web-tools/browser`](./browser-automation.md): Browser automation tools and quarantine lifecycle.
|
|
246
246
|
- [`@arnilo/prism-ag-ui`](./ag-ui.md): Agent-User Interface projection and timeline components.
|
|
247
247
|
- [`@arnilo/prism-core/governance/observability`](./observability.md): OpenTelemetry instrumentation and trace adapters.
|
package/docs/document-reader.md
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
|
-
# Document reader (`@arnilo/prism-
|
|
1
|
+
# Document reader (`@arnilo/prism-work/document-reader`)
|
|
2
2
|
|
|
3
|
-
> **Optional peer install:** `pdf-parse` and/or `mammoth` — see [Optional peer dependencies](peer-dependencies.md).
|
|
3
|
+
> **Optional peer install:** `pdf-parse` and/or `mammoth` — see [Optional peer dependencies](peer-dependencies.md). OCR uses **native fetch**, not an SDK peer.
|
|
4
4
|
|
|
5
5
|
## What it does
|
|
6
6
|
|
|
7
|
-
Optional bounded literal-text extraction for PDF and
|
|
7
|
+
Optional bounded literal-text extraction for PDF, DOCX, XLSX, and PPTX files, consumed by the coding `read` tool. `createDocumentReader()` returns a `DocumentReader` that the host wires into `createReadTool(cwd, { documentReader })`; the read tool then extracts text from supported documents instead of falling back to the raw text page. XLSX renders as TSV and PPTX as a slide title/bullet outline through the in-package OOXML parser. Scanned PDFs/images need a **host-selected** `createMistralOcrParser({ apiKey })` passed in `parsers` — default wiring never calls an external OCR service.
|
|
8
8
|
|
|
9
9
|
## When to use it
|
|
10
10
|
|
|
11
|
-
Use when coding agents must read PDF/Office files (specs, requirements docs, reports) as literal text. Do **not** use it when embedded content execution, macro evaluation, or external resource fetching is required — this adapter never does any of those by construction
|
|
11
|
+
Use when coding agents must read PDF/Office files (specs, requirements docs, reports) as literal text. Do **not** use it when embedded content execution, macro evaluation, or external resource fetching is required — this adapter never does any of those by construction. PDF/DOCX use optional peer parsers (`pdf-parse`, `mammoth`); XLSX/PPTX use the bounded Prism OOXML parser. Docker-less hosts that need document reads pair this with the network-free native sandbox backend (`@arnilo/prism-coding-tools/security` `createNativeSandbox`) for the surrounding tool execution.
|
|
12
12
|
|
|
13
13
|
Activation is explicit: no file-extension sniffing anywhere enables parsing. Absent `documentReader` option = exactly the 0.1.5 read behavior.
|
|
14
14
|
|
|
@@ -21,10 +21,10 @@ Activation is explicit: no file-extension sniffing anywhere enables parsing. Abs
|
|
|
21
21
|
| `maxBytes` | Hard input size cap; oversize files refuse before loading | 32 MiB | 512 MiB |
|
|
22
22
|
| `maxPages` | Page cap for formats that report pages; over-page documents refuse | 1000 | 10 000 |
|
|
23
23
|
| `maxTextBytes` | Extracted-literal-text cap; over-cap results truncate (`truncatedBy: "bytes"`) | 2 MiB | 64 MiB |
|
|
24
|
-
| `parsers` | Host-selected `DocumentParser[]`; default wiring
|
|
24
|
+
| `parsers` | Host-selected `DocumentParser[]`; default wiring includes peer-backed PDF/DOCX plus OOXML XLSX/PPTX | `[pdf, docx, xlsx, pptx]` | — |
|
|
25
25
|
| `redactor` | Optional `SecretRedactor` applied to extracted text at the adapter boundary | none | — |
|
|
26
26
|
|
|
27
|
-
Format gating is magic-byte based: PDF header (`%PDF-`); DOCX zip container
|
|
27
|
+
Format gating is magic-byte based: PDF header (`%PDF-`); DOCX/XLSX/PPTX zip container plus `word/document.xml`, `xl/workbook.xml`, or `ppt/presentation.xml` part marker. Unsupported buffers return `null` and the read falls through to its text path.
|
|
28
28
|
|
|
29
29
|
## Outputs / response / events
|
|
30
30
|
|
|
@@ -36,7 +36,7 @@ Errors: `DocumentReaderError` with code `ERR_PRISM_DOCUMENT_READER` for missing
|
|
|
36
36
|
|
|
37
37
|
```ts
|
|
38
38
|
import { createReadTool } from "@arnilo/prism-coding-tools/agent";
|
|
39
|
-
import { createDocumentReader } from "@arnilo/prism-
|
|
39
|
+
import { createDocumentReader } from "@arnilo/prism-work/document-reader";
|
|
40
40
|
|
|
41
41
|
const documentReader = await createDocumentReader({
|
|
42
42
|
maxBytes: 32 * 1024 * 1024,
|
|
@@ -51,7 +51,7 @@ A `read` of `spec.pdf` yields text content extracted from the PDF (up to 2 MiB o
|
|
|
51
51
|
## Implementation example
|
|
52
52
|
|
|
53
53
|
```ts
|
|
54
|
-
import { createDocumentReader, createPdfParser, type DocumentParser } from "@arnilo/prism-
|
|
54
|
+
import { createDocumentReader, createPdfParser, type DocumentParser } from "@arnilo/prism-work/document-reader";
|
|
55
55
|
|
|
56
56
|
// Host-selected parser wiring: swap in a different PDF backend without touching bounds.
|
|
57
57
|
const myPdfParser: DocumentParser = {
|
|
@@ -63,18 +63,26 @@ const myPdfParser: DocumentParser = {
|
|
|
63
63
|
},
|
|
64
64
|
};
|
|
65
65
|
const reader = await createDocumentReader({ parsers: [myPdfParser, await createPdfParser()] });
|
|
66
|
+
|
|
67
|
+
import { createMistralOcrParser } from "@arnilo/prism-work/document-reader";
|
|
68
|
+
const ocr = createMistralOcrParser({
|
|
69
|
+
apiKey: hostKey, // never read from process.env
|
|
70
|
+
recordUsage: (u) => router.recordUsage({ /* Task 7 */ tokens: 0, costUsd: hostPrice(u) }),
|
|
71
|
+
});
|
|
72
|
+
const scanned = await createDocumentReader({ parsers: [ocr] }); // not in the default parser list
|
|
66
73
|
```
|
|
67
74
|
|
|
68
75
|
## Extension and configuration notes
|
|
69
76
|
|
|
70
|
-
- Default parser wiring uses
|
|
77
|
+
- Default parser wiring uses optional peer dependencies `pdf-parse` (PDF) and `mammoth` (DOCX raw text), plus in-package XLSX/PPTX parsing. Missing selected peers fail closed at creation — never at read time. XLSX sheet count and PPTX slide count share `maxPages`; text remains capped at `maxTextBytes`.
|
|
78
|
+
- `createMistralOcrParser` is **not** a default parser. It POSTs `https://api.mistral.ai/v1/ocr` (`mistral-ocr-latest`) with inline `data:` URLs (`include_image_base64: false`). No Files API upload, so no remote cleanup. Host `documentUrl` values pass `assertSsrfAllowedUrl`. Extracted markdown is untrusted. Caps: 8 MiB / 32 pages / 60 s / 1 in-flight by default (hard 50 MiB / 10 000 pages / 180 s / 4). Pass `recordUsage` to admit cost through Task 7 accounting. `baseUrl` selects residency.
|
|
71
79
|
- DOCX has no page concept in raw text: `pages` is always `1` and the page cap applies to PDF only; the text cap governs DOCX output.
|
|
72
80
|
- The read tool re-checks `maxTextBytes` on results (parity with its text-page bounds check) and refuses reader output beyond it.
|
|
73
81
|
- The adapter truncates over-cap text at a UTF-8 byte boundary (never splits a code point).
|
|
74
82
|
|
|
75
83
|
## Security and performance notes
|
|
76
84
|
|
|
77
|
-
- No embedded-script execution, no macro evaluation, no external resource fetching — the peer raw-text surfaces are pure extractors, and the no-fetch property is enforced by an egress tripwire test.
|
|
85
|
+
- No embedded-script execution, no macro evaluation, no external resource fetching on the **default** parsers — the peer raw-text surfaces are pure extractors, and the no-fetch property is enforced by an egress tripwire test on `document-reader/index.js`. OCR is a separate module and only runs when the host passes that parser.
|
|
78
86
|
- Decompression/size-bomb protection: the read tool stats and refuses files above `maxBytes` before loading; output is capped at `maxTextBytes`.
|
|
79
87
|
- Extraction envelope (recorded in `scripts/budgets.json` `docReader`, measured 2026-08-11): a max-cap 1000-page PDF (288 KB) extracts in ~162 ms with ~17 MB heap delta; the gate asserts completion within the ceiling or documented refusal.
|
|
80
88
|
- Parser code never receives a buffer whose format gate failed; random binaries never reach a parser.
|