@arnilo/prism 0.7.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +73 -0
- package/README.md +12 -11
- package/dist/agent-approval.d.ts +15 -2
- package/dist/agent-approval.js +5 -1
- package/dist/agent-event-source.d.ts +9 -1
- package/dist/agent-event-source.js +10 -3
- package/dist/agent-loops.js +7 -4
- package/dist/agent-run-lifecycle.d.ts +15 -1
- package/dist/agent-run-lifecycle.js +91 -10
- package/dist/agent-run-state.d.ts +34 -2
- package/dist/agent-run-state.js +68 -6
- package/dist/agent-session/helpers.js +20 -1
- package/dist/agent-session/session/assemble.js +250 -27
- package/dist/agent-session/session/persist.d.ts +27 -0
- package/dist/agent-session/session/persist.js +94 -12
- package/dist/agent-session/session/provider-round.d.ts +14 -4
- package/dist/agent-session/session/provider-round.js +197 -25
- package/dist/agent-session/session/tool-round.js +24 -2
- package/dist/agent-session/session/types.d.ts +36 -2
- package/dist/agent-session/session.d.ts +40 -4
- package/dist/agent-session/session.js +78 -5
- package/dist/attention-compiler.d.ts +51 -2
- package/dist/attention-compiler.js +282 -21
- package/dist/cache-helpers.d.ts +4 -2
- package/dist/cache-helpers.js +8 -6
- package/dist/checkpoint-restore.d.ts +45 -0
- package/dist/checkpoint-restore.js +54 -0
- package/dist/checkpoints.js +7 -11
- package/dist/context-budget.d.ts +2 -1
- package/dist/context-budget.js +24 -2
- package/dist/contracts-core/agent.d.ts +30 -0
- package/dist/contracts-core/attention.d.ts +95 -0
- package/dist/contracts-core/content.d.ts +15 -0
- package/dist/contracts-core/guardrail-packs.d.ts +41 -0
- package/dist/contracts-core/guardrail-packs.js +2 -0
- package/dist/contracts-core/loop.d.ts +42 -0
- package/dist/contracts-core/provider.d.ts +25 -0
- package/dist/contracts-core/run-limits.d.ts +21 -0
- package/dist/contracts-core/session.d.ts +23 -5
- package/dist/contracts-core/session.js +21 -2
- package/dist/contracts-core/usage.d.ts +40 -0
- package/dist/contracts-core/usage.js +8 -0
- package/dist/contracts-core.d.ts +2 -0
- package/dist/contracts-core.js +2 -0
- package/dist/contracts-protocol.d.ts +90 -4
- package/dist/contracts-run-state.d.ts +82 -6
- package/dist/evidence-grounding.d.ts +29 -0
- package/dist/evidence-grounding.js +162 -0
- package/dist/guardrail-packs/coding-standard.d.ts +3 -0
- package/dist/guardrail-packs/coding-standard.js +63 -0
- package/dist/guardrail-packs/destructive-commands.d.ts +3 -0
- package/dist/guardrail-packs/destructive-commands.js +46 -0
- package/dist/guardrail-packs/errors.d.ts +7 -0
- package/dist/guardrail-packs/errors.js +9 -0
- package/dist/guardrail-packs/index.d.ts +4 -0
- package/dist/guardrail-packs/index.js +15 -0
- package/dist/guardrail-packs/secrets-hygiene.d.ts +3 -0
- package/dist/guardrail-packs/secrets-hygiene.js +23 -0
- package/dist/guardrail-packs/types.d.ts +16 -0
- package/dist/guardrail-packs/types.js +2 -0
- package/dist/guardrail-packs/validation-respect.d.ts +3 -0
- package/dist/guardrail-packs/validation-respect.js +53 -0
- package/dist/guardrails.d.ts +20 -1
- package/dist/guardrails.js +268 -0
- package/dist/host-composition.d.ts +13 -0
- package/dist/host-composition.js +33 -2
- package/dist/index.d.ts +19 -10
- package/dist/index.js +11 -6
- package/dist/input.d.ts +8 -1
- package/dist/input.js +68 -6
- package/dist/middleware.d.ts +37 -2
- package/dist/middleware.js +41 -0
- package/dist/node/session-store-jsonl.js +18 -3
- package/dist/observability.js +6 -0
- package/dist/provider-events.d.ts +11 -3
- package/dist/provider-events.js +62 -4
- package/dist/providers/openai-compatible.js +6 -3
- package/dist/providers/transport.d.ts +3 -1
- package/dist/providers/transport.js +36 -0
- package/dist/redaction.js +18 -2
- package/dist/run-bundle.d.ts +89 -0
- package/dist/run-bundle.js +150 -0
- package/dist/run-limits.d.ts +11 -1
- package/dist/run-limits.js +46 -0
- package/dist/session-stores.d.ts +12 -1
- package/dist/session-stores.js +21 -4
- package/dist/testing/agent-event-source-conformance.js +41 -2
- package/dist/testing/prefix-stability-conformance.d.ts +30 -0
- package/dist/testing/prefix-stability-conformance.js +104 -0
- package/dist/testing/session-store-conformance.d.ts +3 -2
- package/dist/testing/session-store-conformance.js +48 -0
- package/dist/testing/state-concurrency-conformance.js +5 -12
- package/dist/tools.d.ts +5 -0
- package/dist/tools.js +11 -3
- package/dist/usage-estimation.d.ts +29 -0
- package/dist/usage-estimation.js +79 -0
- package/docs/ag-ui.md +5 -0
- package/docs/agent-events.md +68 -1
- package/docs/agent-loops.md +33 -0
- package/docs/agent-session-runtime.md +5 -3
- package/docs/attention-compiler.md +89 -8
- package/docs/coding-agent-tools.md +1 -1
- package/docs/coding-security.md +1 -0
- package/docs/coding-tools.md +0 -1
- package/docs/compaction-and-retry.md +1 -1
- package/docs/compaction-observational-memory.md +34 -7
- package/docs/connected-apps.md +116 -0
- package/docs/context-and-skills.md +13 -0
- package/docs/core.md +1 -1
- package/docs/diagrams.md +6 -6
- package/docs/document-reader.md +9 -9
- package/docs/documents.md +32 -11
- package/docs/durable-runs.md +129 -0
- package/docs/embeddings.md +5 -0
- package/docs/enterprise-postgres-state.md +4 -0
- package/docs/evaluations.md +5 -0
- package/docs/execution-timeline.md +84 -1
- package/docs/guardrails.md +71 -2
- package/docs/history/079-messaging-primitive-review.md +391 -0
- package/docs/history/080-messaging-followon-primitive-review.md +234 -0
- package/docs/history/081-connected-apps-primitive-review.md +74 -0
- package/docs/history/083-prism-work-primitive-review.md +84 -0
- package/docs/history/084-primitive-review.md +96 -0
- package/docs/history/085-honesty-and-cut-primitive-review.md +91 -0
- package/docs/history/README.md +5 -0
- package/docs/history/release-handoffs.md +38 -0
- package/docs/host-compositions.md +8 -6
- package/docs/host-security.md +2 -2
- package/docs/index.md +66 -29
- package/docs/input-and-prompt-assembly.md +3 -3
- package/docs/knowledge-sync.md +4 -0
- package/docs/live-testing.md +5 -3
- package/docs/mcp-tools.md +1 -0
- package/docs/messaging-channel-operations.md +166 -0
- package/docs/messaging-channels.md +150 -0
- package/docs/middleware-hooks.md +38 -2
- package/docs/migrate-to-0.8.md +124 -0
- package/docs/migrate-to-0.9.md +210 -0
- package/docs/migration.md +43 -0
- package/docs/model-registry.md +12 -2
- package/docs/multi-agent-patterns.md +25 -2
- package/docs/node-jsonl-session-store.md +7 -1
- package/docs/observability.md +7 -3
- package/docs/openapi-tools.md +1 -1
- package/docs/operations.md +1 -3
- package/docs/options-index.md +36 -3
- package/docs/peer-dependencies.md +6 -6
- package/docs/policy-and-audit.md +13 -1
- package/docs/postgres-persistence.md +1 -1
- package/docs/prefix-stability-conformance.md +93 -0
- package/docs/provider-caching.md +4 -4
- package/docs/provider-conformance.md +16 -0
- package/docs/provider-layer.md +2 -2
- package/docs/provider-packages.md +20 -20
- package/docs/providers/neuralwatt.md +5 -1
- package/docs/public-contracts.md +2 -2
- package/docs/rag.md +102 -4
- package/docs/release-and-install.md +55 -47
- package/docs/run-bundle.md +92 -0
- package/docs/runs-and-usage.md +57 -6
- package/docs/scoped-agent-memory.md +262 -0
- package/docs/server.md +2 -0
- package/docs/session-store-conformance.md +1 -2
- package/docs/session-stores.md +17 -17
- package/docs/sheets.md +9 -9
- package/docs/signal-channel.md +112 -0
- package/docs/speech.md +5 -1
- package/docs/sqlite-persistence.md +1 -1
- package/docs/supervisors.md +32 -12
- package/docs/telegram-channel.md +157 -0
- package/docs/testing.md +2 -2
- package/docs/tools.md +17 -0
- package/docs/wiki.md +1 -1
- package/docs/work-artifacts-and-review.md +1 -1
- package/docs/work-connectors.md +9 -9
- package/docs/work-sandbox.md +115 -0
- package/docs/work-tools.md +38 -16
- package/docs/workflows.md +5 -0
- package/package.json +9 -3
- package/templates/business-worker/manifest.json +2 -1
- package/templates/business-worker/src/agent.ts.tmpl +1 -1
- package/templates/business-worker/src/tests/agent.test.ts.tmpl +1 -1
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# Messaging channels
|
|
2
|
+
|
|
3
|
+
## What it does
|
|
4
|
+
|
|
5
|
+
`@arnilo/prism-channels` provides transport-neutral contracts and the Prism execution adapter for chat transports. A host authorizes an observed sender, binds them to an owned Prism session, and the runtime runs ordinary turns through the existing `AgentSession` API and projects only the current run's final text back to the bound destination.
|
|
6
|
+
|
|
7
|
+
The root export ships the runtime, contracts, limits and durable journal. The official Telegram adapter ships as `@arnilo/prism-channels/telegram`; experimental Signal support ships as the isolated Node-only `@arnilo/prism-channels/signal` subpath. Durable single-decision approval controls ship now.
|
|
8
|
+
|
|
9
|
+
Pass `checkpoints` (and optionally `leases`) to make bindings, admission dedup, the claim before provider work and outgoing replies durable; that is the restart-safe deployment. Without a store the runtime keeps bounded in-process state only — still the correct execution/authorization seam, but restart-unsafe. Recovery mechanics, record layout and the operator runbook: [Messaging channel operations](messaging-channel-operations.md).
|
|
10
|
+
|
|
11
|
+
## When to use it
|
|
12
|
+
|
|
13
|
+
Use when a host already operates a chat bot (or plans to) and wants selected Prism agents reachable by chat message without building another agent runtime or conversation database. Telegram admits private DMs by default and group/topic text only when the adapter opts in with `allowGroups` and the host grants the exact chat/thread/sender; Signal (experimental) stays direct-message only. Not for media, voice or proactive broadcasting in the current scope.
|
|
14
|
+
|
|
15
|
+
## Inputs / request
|
|
16
|
+
|
|
17
|
+
Install the required `@arnilo/prism` peer plus this package. `createMessagingRuntime` takes `MessagingRuntimeOptions`: `authorize`, `resolveAgent`, `deliver`, optional `limits` (`ChannelLimits`), `redactor`, `checkpoints`, `leases`, and `resolveBinding`. Durable writers are also constructed with `ChannelStateStoreOptions`, `ChannelDeliveryJournalOptions`, and `ChannelPairingStoreOptions`.
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
npm install @arnilo/prism @arnilo/prism-channels
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Request/response example
|
|
24
|
+
|
|
25
|
+
```ts
|
|
26
|
+
import { createMessagingRuntime } from "@arnilo/prism-channels";
|
|
27
|
+
|
|
28
|
+
const runtime = createMessagingRuntime({
|
|
29
|
+
// Observed sender + action -> host-verified grant, or `false`.
|
|
30
|
+
authorize: ({ connectionId, externalActorId, action }) => hostGrants.lookup(connectionId, externalActorId, action),
|
|
31
|
+
// Authorized alias -> the host-configured agent for the current identity/ownership.
|
|
32
|
+
resolveAgent: ({ agentAlias, identity, ownership }) => hostAgents.resolve(agentAlias, identity, ownership),
|
|
33
|
+
// Delivery seam: normally the transport adapter's `send`, or a durable outbox.
|
|
34
|
+
deliver: (reply) => transport.send(reply),
|
|
35
|
+
// Optional preview seam: cumulative redacted assistant text while a turn runs (Telegram drafts).
|
|
36
|
+
onAssistantDelta: (delta) => transport.sendDraft?.(delta),
|
|
37
|
+
// Optional media seam: normally the adapter's own bounded `fetchAttachment`.
|
|
38
|
+
// An image becomes model input only when the resolved model declares image input.
|
|
39
|
+
fetchAttachment: (ref) => transport.fetchAttachment(ref),
|
|
40
|
+
limits: { maxPendingPerBinding: 4 },
|
|
41
|
+
redactor: hostRedactor,
|
|
42
|
+
// Durable journal (optional but required for restart safety): any CheckpointStore/LeaseStore,
|
|
43
|
+
// for example createMemoryCheckpointStore(), SQLite or PostgreSQL persistence.
|
|
44
|
+
checkpoints: hostCheckpoints,
|
|
45
|
+
leases: hostLeases,
|
|
46
|
+
});
|
|
47
|
+
|
|
48
|
+
// The adapter calls this before acknowledging transport receipt (Telegram offset / webhook 2xx).
|
|
49
|
+
const admission = await runtime.admit(event);
|
|
50
|
+
await runtime.drain({ deadlineMs: 5_000 }); // host shutdown / test settle
|
|
51
|
+
runtime.diagnostics(); // bounded counters, no text or external ids
|
|
52
|
+
await runtime.stop();
|
|
53
|
+
|
|
54
|
+
// Durable recovery, host-driven and never automatic:
|
|
55
|
+
await runtime.listUnresolved({ identity }); // bounded page: operations still needing attention
|
|
56
|
+
await runtime.reconcile({ identity, connectionId, operationId, expectedVersion, acknowledgeDuplicateRisk: true });
|
|
57
|
+
await runtime.prune({ identity }); // retention sweep; unresolved work is never deleted
|
|
58
|
+
|
|
59
|
+
// Opt-in host notice to one already-bound pair (no agent run, no tools, no queue):
|
|
60
|
+
// the grant must carry `notifications: true`, `notifyId` is the idempotency key.
|
|
61
|
+
await runtime.notify({
|
|
62
|
+
identity,
|
|
63
|
+
connectionId: "telegram-main",
|
|
64
|
+
externalConversationId: "8241",
|
|
65
|
+
notifyId: "job-123-done",
|
|
66
|
+
text: "The nightly job finished.",
|
|
67
|
+
});
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Outputs / response / events
|
|
71
|
+
|
|
72
|
+
`admit` authorizes first and returns `{ status: "accepted" | "denied" | "unsupported", reason?, operationId?, duplicate? }`. It resolves before any provider call and before the transport acknowledgment; accepted turns then run in the background.
|
|
73
|
+
|
|
74
|
+
`ChannelInboundEvent` carries only transport-observed fields: `connectionId`, `externalConversationId`, `externalActorId` (stable platform id, never a display name), `eventId` (the transport acknowledgment unit), `text`, optional `threadId`, `attachments`, `claims`, and an adapter-parsed opaque `approval` control. `claims` and transport ids are untrusted context for the host callback only — never authority.
|
|
75
|
+
|
|
76
|
+
`attachments` are bounded `ChannelAttachmentRef` values (`kind: "image" | "document" | "voice"`, transport file id, optional mime/size/name) — identifiers, never bytes and never model-selected. Voice and document become turn text in the adapter through host hooks (`transcribe`, `extractDocumentText`); only an `image` can become model input, and only when the resolved agent's model declares `image` input *and* the host wired `MessagingRuntimeOptions.fetchAttachment` (normally the adapter's `fetchAttachment`). Then the runtime fetches the current event's refs once, bounded by the channel cap, builds one image content block per ref next to the text, and stores nothing: an undeclared modality, a missing seam or a failed fetch produces `unsupported_media` with a bounded notice and no provider call. Declared attachment bytes above `maxAttachmentBytes` are denied (`reason: "oversized"`) before any fetch.
|
|
77
|
+
|
|
78
|
+
`MessagingRuntimeOptions.onAssistantDelta` is the optional outbound preview seam: while a turn runs, the runtime forwards the cumulative assistant text of the current message, already redacted with the reply `redactor`, as `ChannelAssistantDelta` (`connectionId`, `externalConversationId`, optional `threadId`, `text`). It fires once per streamed text chunk, only for turns that actually reach the provider, stops at cancel or `stop()`, and resets per assistant message so a tool-call preamble never leaks into the following answer. It is not a reply: nothing is journaled, delivery is not tracked, a throwing callback is ignored, and the terminal `ChannelReply` is unchanged. An adapter that has no preview capability simply ignores it.
|
|
79
|
+
|
|
80
|
+
`MessagingRuntime.notify` is the opt-in reverse direction: the host sends a `notice` (never a `final`) to one already-bound conversation/thread pair, without a model run, a tool call or a queue slot. It takes `identity`, the bound destination and a caller `notifyId`; the grant must be re-checked with `action: "notify"` and carry `notifications: true`, and `notifyId` is the idempotency key (`duplicate: true` on repeat). The notice is redacted, bounded by the reply cap, journaled as an operation of kind `notify` and staged through the same reply journal as an answer — so a crash between the answer and the send never turns into a silent notice and never re-runs anything. There is no broadcast or fan-out input: an unbound pair, another actor's binding, a foreign ownership scope, an ungranted or revoked identity and an unresolvable binding all fail closed with a bounded `reason`.
|
|
81
|
+
|
|
82
|
+
## Implementation example
|
|
83
|
+
|
|
84
|
+
Network-free host composition: [`examples/messaging-agent.ts`](../examples/messaging-agent.ts). Telegram: [`examples/telegram-agent.ts`](../examples/telegram-agent.ts). Experimental Signal: [`examples/signal-agent.ts`](../examples/signal-agent.ts).
|
|
85
|
+
|
|
86
|
+
## Security and performance notes
|
|
87
|
+
|
|
88
|
+
### Authorization
|
|
89
|
+
|
|
90
|
+
- `authorize` receives the bounded text and action (`"message"`, `"command"`, or `"approval"`) and must return a host-verified `AgentIdentity` (`verified: true`, non-empty scopes, unexpired, unrevoked) plus permitted `agentAliases` and a `grantRevision`. Anything else fails closed with a bounded `reason` and no provider call.
|
|
91
|
+
- Ownership is always derived with `ownershipFromIdentity`; the event, message text and `claims` cannot set or widen it. The run receives the verified `identity` and the runtime never passes a `per-run ownership` override, so `createSecureAgent` defaults cannot be replaced by a channel adapter.
|
|
92
|
+
- Every turn re-checks the grant and `assertIdentityActive` before provider work and before delivery: a revoked identity neither runs nor sends, and no reply is sent after revocation.
|
|
93
|
+
- Unknown aliases, malformed ids, oversized text, ungranted senders and capacity pressure are denied without provider or tool calls. Cross-tenant errors stay bounded and never reveal whether a foreign session exists.
|
|
94
|
+
- **Media boundary.** Attachment refs are validated (count, ids, metadata, safe-integer sizes) and malformed refs are denied `malformed`; sizes above `maxAttachmentBytes` are denied `oversized` before a fetch. The runtime fetches only refs of the event it is running and passes bytes straight into one run call — the same address the model cannot influence. Nothing media-shaped is written to the journal or the delivery seam: operation and reply records stay text-only, and an attachment the host did not opt into produces a bounded notice rather than a prompt-injection surface.
|
|
95
|
+
|
|
96
|
+
### Execution semantics
|
|
97
|
+
|
|
98
|
+
- **Binding.** Connection + conversation + thread + actor + alias maps to a deterministic owned session id (`chan-<sha256 prefix>`); the host may supply `resolveBinding` to persist its own mapping. Same display name on another connection or tenant is a different session; `/new` starts a fresh session for the current binding. With `checkpoints`, the binding — including the `/new` generation, the branch leaf and a suspended marker — is journaled and reused after a restart.
|
|
99
|
+
- **Serialization.** Turns are serialized per logical session id and bounded per binding, per process and by active-session concurrency. Core sessions do not serialize two runtime objects that share an id, so a channel-only mutex is not shared-session arbitration: if another API can drive the same session, use a shared host admission/lease mechanism instead.
|
|
100
|
+
- **Host notices.** `notify` never allocates a binding or a route and never touches the turn queue; it requires an existing binding and re-checks identity, ownership, the `notifications: true` grant flag and the bound alias on every call. When the host has more than one binding for the same pair (the conversation moved alias), the current selection is addressed and anything ambiguous fails closed. Notices count against `maxPendingPerProcess` and reuse the staged-reply path, so a delivery failure is recorded rather than retried.
|
|
101
|
+
- **Output.** Only the current run's terminal `succeeded` text is delivered. Failed, aborted or suspended runs send a bounded notice (redacted), never a previous turn's text and never partial deltas. Responses are truncated to the channel response cap.
|
|
102
|
+
- **Cancellation.** `/cancel` aborts an active run via its `AbortSignal` and marks queued turns cancelled. For a suspended durable run it submits core's CAS-checked terminal denial; an already dispatched tool is never rolled back. Transport disconnection is not user cancellation; `stop()` marks queued turns cancelled and aborts active runs on host shutdown.
|
|
103
|
+
- **Suspension and approval.** A suspended tool decision blocks ordinary turns (`reason: "awaiting_decision"`) and `/new` cannot bypass it. The runtime persists two short-lived opaque controls for the first pending tool decision: `allow_once` and `reject_once`. Each is bound to exact principal, connection, conversation/thread, agent alias/revision, session, run, approval id and checkpoint version. It is consumed atomically before resume; stale, expired, forwarded, replayed or revoked controls fail closed. Resume reauthorizes the sender and resolves current agent policy before core dispatches the decision. Subsequent pending decisions re-suspend and issue fresh controls. Unsupported elicitation stays blocked for authenticated host handling.
|
|
104
|
+
- **Controls.** `/help`, `/status`, `/new`, `/agent <alias>` and `/cancel` are the only ordinary commands. `/status` reports only the bound operation state; controls bypass the prompt queue. The experimental Signal adapter maps `/approve <token>` and `/deny <token>` to the same opaque records. Any other slash input (including registry/config mutation) never reaches a model.
|
|
105
|
+
- **Durable intake.** With `checkpoints`, every authorized event is recorded before it is acknowledged, deduplicated by connection + event id across processes, and moved `accepted` → `executing` by a compare-and-swap claim *before* the provider is called. A duplicate event is acknowledged with `duplicate: true` and never runs twice; a denial leaves no record behind, so a legitimate retry still works.
|
|
106
|
+
- **Durable replies.** A reply is persisted before `deliver` runs, so a crash between the answer and the send never loses it and never reruns the model. An ambiguous send (`deliver` threw) is recorded as `delivery_unknown` and is never resent automatically; only an authorized `reconcile` call with an explicit `acknowledgeDuplicateRisk` resends the persisted text.
|
|
107
|
+
- **Crash truth.** A claimed operation whose process died stays unresolved and is never replayed; `reconcile` files it as `execution_unknown`, which is a dead-letter for host review, not a retry. There is no exactly-once claim: side effects already accepted by the platform (or already dispatched tools) are not retractable.
|
|
108
|
+
|
|
109
|
+
## Extension and configuration notes
|
|
110
|
+
|
|
111
|
+
`resolveChannelLimits` rejects `ChannelLimits` values outside `[1, hard cap]`.
|
|
112
|
+
|
|
113
|
+
| Limit | Default | Hard cap |
|
|
114
|
+
| --- | --- | --- |
|
|
115
|
+
| `maxInputBytes` | 32 KiB | 64 KiB |
|
|
116
|
+
| `maxResponseBytes` | 64 KiB | 128 KiB |
|
|
117
|
+
| `maxPendingPerBinding` | 8 | 32 |
|
|
118
|
+
| `maxPendingPerProcess` | 100 | 500 |
|
|
119
|
+
| `maxActiveSessions` | 4 | 16 |
|
|
120
|
+
| `maxRoutes` | 1000 | 10000 |
|
|
121
|
+
| `maxSeenEventsPerConnection` | 1000 | 10000 |
|
|
122
|
+
| `stopDeadlineMs` | 30 s | 5 min |
|
|
123
|
+
| `retentionDays` | 7 | 90 |
|
|
124
|
+
| `maxJournalPage` | 100 | 500 |
|
|
125
|
+
| `maxJournalRecordBytes` | 128 KiB | 512 KiB |
|
|
126
|
+
| `leaseTtlMs` | 30 s | 5 min |
|
|
127
|
+
| `approvalTtlMs` | 5 min | 5 min |
|
|
128
|
+
| `maxAttachmentBytes` | 1 MiB | 4 MiB |
|
|
129
|
+
|
|
130
|
+
Capacity pressure fails admission (`reason: "capacity"`) instead of evicting accepted work, and an unusable journal fails admission with `reason: "unavailable"` rather than running unclaimed work. Without `checkpoints` the in-process seen-event set is bounded and is not a durable deduplication guarantee.
|
|
131
|
+
|
|
132
|
+
## Telegram
|
|
133
|
+
|
|
134
|
+
Use [`@arnilo/prism-channels/telegram`](telegram-channel.md) for explicit long polling or a separately mounted webhook handler. Both require a service-owned receiver lease; polling also persists its offset after `admit` settles. The adapter uses only native `fetch`, resolves the bot token at each transport request, never follows redirects, sends plain text in Unicode-safe chunks, and records timeout/network sends as ambiguous so the durable reply journal never blindly resends them. Private DMs are the default; `allowGroups: true` adds `group`/`supergroup` text (forum topics keep their topic id as `threadId`) and only ever hands observed `(chat, thread?, sender)` tuples to the host `authorize` — membership is never an identity.
|
|
135
|
+
|
|
136
|
+
## Signal (experimental)
|
|
137
|
+
|
|
138
|
+
Use [`@arnilo/prism-channels/signal`](signal-channel.md) only with an externally supervised, pinned signal-cli v0.14.8 private Unix socket and an explicit operator acceptable-use/GPL policy attestation. It admits only direct UUID text envelopes after durable writer readiness, pauses on account/storage/lease trouble, sends only to the authorized bound UUID, and does not claim lossless receive or end-to-end encryption through Prism/model providers. Group envelopes are always dropped (no Signal group or topic support).
|
|
139
|
+
|
|
140
|
+
A network-free host composition is `examples/messaging-agent.ts` (sqlite journal, mock agent, `drain`/`stop`). Telegram polling/webhook helpers live in `examples/telegram-agent.ts`; the experimental Signal helper is `examples/signal-agent.ts`. Opt-in live probes skip without credentials — see [Live and end-to-end testing](live-testing.md).
|
|
141
|
+
|
|
142
|
+
## Related APIs
|
|
143
|
+
|
|
144
|
+
- [Telegram channel](telegram-channel.md) — polling, webhook mounting, credentials, limits and failure behavior.
|
|
145
|
+
- [Signal channel (experimental)](signal-channel.md) — daemon prerequisites, policy gate, manual subscription and reliability limits.
|
|
146
|
+
- [Messaging channel operations](messaging-channel-operations.md) — durable records, recovery, reconciliation and the operator runbook.
|
|
147
|
+
- [Agent session runtime](agent-session-runtime.md) — `AgentSession.run`, results, interruption and resume.
|
|
148
|
+
- [Agent identity](agent-identity.md) — `AgentIdentity`, `IdentityVerifier`, ownership projection.
|
|
149
|
+
- [Host security](host-security.md) — host-owned authority and side-effect policy.
|
|
150
|
+
- [Package README](../packages/prism-channels/README.md) — package boundary and peer-only install.
|
package/docs/middleware-hooks.md
CHANGED
|
@@ -14,7 +14,7 @@ APIs:
|
|
|
14
14
|
|
|
15
15
|
Use middleware hooks when a host wants extension/package code to observe or transform a value at a named runtime boundary.
|
|
16
16
|
|
|
17
|
-
Do not use middleware hooks as a provider adapter, prompt builder, retry policy, compaction strategy, tool dispatcher, permission system, or agent/session runtime.
|
|
17
|
+
Do not use middleware hooks as a provider adapter, prompt builder, retry policy, compaction strategy, tool dispatcher, permission system, or agent/session runtime. Per-turn tool menus use `AgentConfig.toolNarrowing` / `RunOptions.toolNarrowing`, not a middleware hook — see [Tools](tools.md).
|
|
18
18
|
|
|
19
19
|
## Inputs / request
|
|
20
20
|
|
|
@@ -24,6 +24,7 @@ createMiddlewareRegistry(options?: MiddlewareRegistryOptions): MiddlewareRegistr
|
|
|
24
24
|
|
|
25
25
|
Built-in hook names:
|
|
26
26
|
|
|
27
|
+
- `beforeProviderTurn`
|
|
27
28
|
- `provider_request`
|
|
28
29
|
- `input_assembly`
|
|
29
30
|
- `prompt_build`
|
|
@@ -47,7 +48,7 @@ Built-in hook names:
|
|
|
47
48
|
|
|
48
49
|
## Outputs / response / events
|
|
49
50
|
|
|
50
|
-
`run()` returns the transformed value. If no middleware is registered for a hook, `run()` returns the original value. `assembleProviderInput()` calls Phase 5 hooks in this order when middleware is supplied: `input_assembly`, then `context`, then `prompt_build`. The `input_assembly` call is unconditional — it runs after whatever `InputBuilder` produced the messages, so host middleware at that hook cannot be skipped by a custom builder. The agent/session runtime applies configured provider request policies, then invokes `provider_request` once with the `ProviderRequest` before `AIProvider.generate()`, invokes `tool_call` and `tool_result` through `dispatchToolCall()` for complete provider tool calls, invokes `compaction` with `{ context, result }` after a compaction strategy returns and before the runtime appends its standard compaction entry, and invokes `retry` with `{ context, decision }` before scheduling a provider-turn retry. There is no `provider_response` hook; observing provider output belongs to the provider adapter or subscriber events.
|
|
51
|
+
`run()` returns the transformed value. If no middleware is registered for a hook, `run()` returns the original value. `assembleProviderInput()` calls Phase 5 hooks in this order when middleware is supplied: `input_assembly`, then `context`, then `prompt_build`. The `input_assembly` call is unconditional — it runs after whatever `InputBuilder` produced the messages, so host middleware at that hook cannot be skipped by a custom builder. The agent/session runtime runs `beforeProviderTurn` once per turn after the request is assembled and before any provider-round work, then applies configured provider request policies, then invokes `provider_request` once with the `ProviderRequest` before `AIProvider.generate()`, invokes `tool_call` and `tool_result` through `dispatchToolCall()` for complete provider tool calls, invokes `compaction` with `{ context, result }` after a compaction strategy returns and before the runtime appends its standard compaction entry, and invokes `retry` with `{ context, decision }` before scheduling a provider-turn retry. There is no `provider_response` hook; observing provider output belongs to the provider adapter or subscriber events.
|
|
51
52
|
|
|
52
53
|
With default `errorPolicy: "event"`, middleware errors become `extension_error` events when `onError` is provided, and later middleware still runs with the current value. With `errorPolicy: "throw"`, `run()` rejects on the first middleware error.
|
|
53
54
|
|
|
@@ -92,11 +93,45 @@ export const extension: Extension = {
|
|
|
92
93
|
};
|
|
93
94
|
```
|
|
94
95
|
|
|
96
|
+
## No-model turns (`beforeProviderTurn`)
|
|
97
|
+
|
|
98
|
+
`beforeProviderTurn` lets the host answer a turn from data it already has — teaching empty states, canned flows, deterministic lookups — without any provider request. The payload is `BeforeProviderTurnPayload` (`sessionId`, `runId`, `turn`, `userText`) and middleware returns it unchanged or with `answer: DeterministicTurnAnswer` set:
|
|
99
|
+
|
|
100
|
+
```ts
|
|
101
|
+
export interface DeterministicTurnAnswer {
|
|
102
|
+
readonly content: readonly ContentBlock[];
|
|
103
|
+
readonly provenance: { readonly middleware: string };
|
|
104
|
+
}
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
```ts
|
|
108
|
+
import { createAgent, createMiddlewareRegistry, type BeforeProviderTurnPayload } from "@arnilo/prism";
|
|
109
|
+
|
|
110
|
+
const DESK_ANSWERS = new Map([["what can you do?", "I answer from local records; ask about an order id."]]);
|
|
111
|
+
const middleware = createMiddlewareRegistry();
|
|
112
|
+
middleware.use<BeforeProviderTurnPayload>("beforeProviderTurn", (payload, next) => {
|
|
113
|
+
const text = DESK_ANSWERS.get(payload.userText);
|
|
114
|
+
return text ? { ...payload, answer: { content: [{ type: "text", text }], provenance: { middleware: "desk" } } } : next(payload);
|
|
115
|
+
});
|
|
116
|
+
|
|
117
|
+
const session = createAgent({ model, provider, middleware }).createSession();
|
|
118
|
+
await session.run("what can you do?"); // no provider call; assistant message recorded
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Contract:
|
|
122
|
+
|
|
123
|
+
- Returning the payload without `answer` (or returning `undefined`) sends the turn to the provider exactly as if the hook were absent.
|
|
124
|
+
- `answer.provenance.middleware` is mandatory and validated as a bounded id (1–64 chars: letters, digits, `.` `_` `:` `-`); a deterministic turn can never masquerade as model output.
|
|
125
|
+
- `answer.content` accepts assistant-visible content blocks (`text`, `image`, `audio`, `file`, `document`, `video`, `thinking`). Tool-call blocks are rejected — no provider ran to authorize a call — and an empty block array throws `DeterministicTurnError` (`ERR_PRISM_DETERMINISTIC_TURN`), failing the run closed instead of falling through to the provider.
|
|
126
|
+
- Content passes the same output guardrails as provider output and is charged against `maxResponseBytes`, but the turn records no usage: usage is absent, never zero, and the run timeline shows a `deterministic` step named after the answering middleware.
|
|
127
|
+
- Provenance persists: the assistant message carries `metadata.deterministic = { middleware }`, so a transcript loaded back from any session store still proves the turn had no model behind it. `summarizeTimeline()`/`summarizeSession()` report `turns: { model, deterministic }`, and `createDeterministicTurnScorer()` (from `@arnilo/prism-core/governance/evals`) grades a trajectory for no-model coverage — failing a turn that both answered deterministically and still issued a provider request.
|
|
128
|
+
|
|
95
129
|
## Extension and configuration notes
|
|
96
130
|
|
|
97
131
|
- Middleware registration is explicit through `createMiddlewareRegistry()` or `ExtensionAPI.use()`.
|
|
98
132
|
- `provider_request` middleware sees generic `ProviderRequest.options` after request policies have run; do not add secrets unless a redactor/policy secret list covers that boundary.
|
|
99
133
|
- Middleware runs only when the host/runtime calls `run()` or passes the registry to a helper that documents a call site.
|
|
134
|
+
- `beforeProviderTurn` runs only for turns that reach the provider boundary; a turn already ended by a run limit, host turn policy, or durable suspension never reaches it, and host middleware is trusted code — it must not use the hook to bypass `RunLimits` or guardrails.
|
|
100
135
|
- `compaction` middleware may adjust the compaction result summary/data, but runtime still owns session store append ordering and branch parent ids.
|
|
101
136
|
- `retry` middleware may stop retrying or adjust delay, but runtime still owns retry event emission, abort-aware waiting, and provider-turn boundaries.
|
|
102
137
|
- The registry does not discover packages, read manifests, load config, call providers, execute tools, read resources, or start sessions.
|
|
@@ -112,6 +147,7 @@ export const extension: Extension = {
|
|
|
112
147
|
|
|
113
148
|
## Related APIs
|
|
114
149
|
|
|
150
|
+
- [Middlewares vs restore hooks](durable-runs.md#restore-hooks-all-or-nothing): middleware transforms payloads at named boundaries; `restoreHooks` restore external state before a durable resume and are not middleware.
|
|
115
151
|
- [Extension kernel and event bus](extensions.md): `ExtensionAPI.use()` and shared error policy.
|
|
116
152
|
- [Contribution registries](contribution-registries.md): direct contribution registration separate from middleware.
|
|
117
153
|
- [Agent/session runtime](agent-session-runtime.md): provider request policy/middleware timing, bounded tool loop call site for `tool_call`/`tool_result` hooks, and runtime call sites for `compaction` and `retry`.
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
# Migrate Prism 0.7 to 0.8
|
|
2
|
+
|
|
3
|
+
> **Status: 0.8.0** (messaging channels, connected apps, work family, durable runs).
|
|
4
|
+
|
|
5
|
+
This document details migration steps, breaking import-map changes, and compatibility notes for upgrading from Prism 0.7.0 to 0.8.0.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Security Tightenings and Breaking Behavioral Changes
|
|
10
|
+
|
|
11
|
+
### 1. Work family package move (the only import-map break)
|
|
12
|
+
|
|
13
|
+
`@arnilo/prism-office` is removed with no pre-1.0 shim. Install `@arnilo/prism-work` next to `@arnilo/prism` and rewrite imports:
|
|
14
|
+
|
|
15
|
+
| Old import | Replacement |
|
|
16
|
+
| --- | --- |
|
|
17
|
+
| `@arnilo/prism-office/documents` | `@arnilo/prism-work/documents` |
|
|
18
|
+
| `@arnilo/prism-office/sheets` | `@arnilo/prism-work/sheets` |
|
|
19
|
+
| `@arnilo/prism-office/diagrams` | `@arnilo/prism-work/diagrams` |
|
|
20
|
+
| `@arnilo/prism-core/integrations/work` | `@arnilo/prism-work/connectors` |
|
|
21
|
+
| `@arnilo/prism-core/integrations/work/microsoft365` | `@arnilo/prism-work/connectors/microsoft365` |
|
|
22
|
+
| `@arnilo/prism-core/integrations/work/google-workspace` | `@arnilo/prism-work/connectors/google-workspace` |
|
|
23
|
+
| `@arnilo/prism-core/integrations/work/drafts` | `@arnilo/prism-work/connectors/drafts` |
|
|
24
|
+
| `@arnilo/prism-coding-tools/document-reader` | `@arnilo/prism-work/document-reader` |
|
|
25
|
+
|
|
26
|
+
`createReadTool({ documentReader })` is unchanged: pass a reader from the new subpath. Core keeps the durable adapter at `@arnilo/prism-core/enterprise/postgres` (`createPostgresEnterpriseState({ pool }).workIdempotency`) with type-only structural coupling.
|
|
27
|
+
|
|
28
|
+
**Match work-idempotency conflicts by `code`, not by error class.** Portable codes stay `ERR_PRISM_WORK_IDEMPOTENCY` and `ERR_PRISM_WORK_IDEMPOTENCY_CONFLICT`. `createMemoryIdempotencyStore()` throws `WorkToolError`; the PostgreSQL adapter throws `EnterprisePostgresError` because `@arnilo/prism-core` cannot depend on `@arnilo/prism-work` at runtime.
|
|
29
|
+
|
|
30
|
+
#### Migration Actions
|
|
31
|
+
- Replace every `@arnilo/prism-office` and `integrations/work` / coding-tools `document-reader` import with the table above.
|
|
32
|
+
- Catch work-idempotency by `error.code`.
|
|
33
|
+
- Install `@arnilo/prism-work@^0.8.0` (it is one of the eleven lockstep packages).
|
|
34
|
+
|
|
35
|
+
### 2. Observational-memory workers stay tool-only
|
|
36
|
+
|
|
37
|
+
Workers (`observer` / `dropper` / `reflector`) keep only `tool_call` provider events. A text-only, thinking-only, or done-only turn is a **successful no-op**: no ledger write from that turn. Mixed text+tools keep the tools. Limit and unknown-tool failures throw `MemoryError` / `MemoryLimitError` (`code`), not an English-prefix match.
|
|
38
|
+
|
|
39
|
+
#### Migration Actions
|
|
40
|
+
- Do not expect assistant prose from a worker turn to become an observation.
|
|
41
|
+
- Catch worker-limit failures with `instanceof MemoryError` (or `MemoryLimitError`) / `error.code`, not message prefix.
|
|
42
|
+
|
|
43
|
+
### 3. Channel lease release is fail-closed
|
|
44
|
+
|
|
45
|
+
`createMessagingRuntime` clears an in-memory route lease only after the lease store acknowledges `releaseLease`. A store throw increments `storageFailures`, leaves the token on the route, and retries on the next idle/`stop` path. TTL remains the cross-process backstop. An already-delivered reply is not rolled back.
|
|
46
|
+
|
|
47
|
+
#### Migration Actions
|
|
48
|
+
- Treat a failed release as “this process still holds the binding,” not as free.
|
|
49
|
+
- Do not log lease tokens.
|
|
50
|
+
|
|
51
|
+
### 4. AG-UI input authority is opt-in server-side
|
|
52
|
+
|
|
53
|
+
`CreateAgUiHandlerOptions.inputPolicy.clientState: "ignore"` validates then discards client-supplied AG-UI state and tools before projection, and stops advertising client-provided tools. Default `"honor"` is byte-identical to 0.7.0.
|
|
54
|
+
|
|
55
|
+
#### Migration Actions
|
|
56
|
+
- Hosts that must not trust the browser for tools/state set `inputPolicy: { clientState: "ignore" }`.
|
|
57
|
+
- Leave the default if the 0.7.0 honor path is intended.
|
|
58
|
+
|
|
59
|
+
### 5. Checkpoint foreign-scope reads no longer leak existence
|
|
60
|
+
|
|
61
|
+
A checkpoint or agent-run status load under a foreign ownership scope is a miss (or a generic CAS conflict), not a distinct “exists but not yours” error. `ERR_PRISM_AGENT_RUN_STATE` covers a missing run and a foreign-scope read.
|
|
62
|
+
|
|
63
|
+
#### Migration Actions
|
|
64
|
+
- Stop catching `Checkpoint ownership mismatch` (or equivalent) as an existence signal.
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## Additive surfaces (inert unless wired)
|
|
69
|
+
|
|
70
|
+
### 6. Messaging channels (`@arnilo/prism-channels`)
|
|
71
|
+
|
|
72
|
+
New eleventh publishable package. Transport-neutral runtime: deny-by-default sender authorization, owned session binding, serialized turns, current-run replies, one-use durable approvals, bounded attachment refs. Official Telegram adapter (private DMs; opt-in granted groups/topics; opt-in streaming drafts in private chats; bounded media; optional voice transcription/synthesis; opt-in notices to one already-bound pair). Experimental Signal adapter (pinned signal-cli, explicit policy gate, UUID DM filtering). See [messaging channels](messaging-channels.md), [Telegram](telegram-channel.md), [Signal](signal-channel.md), [operations](messaging-channel-operations.md).
|
|
73
|
+
|
|
74
|
+
#### Migration Actions
|
|
75
|
+
- Install `@arnilo/prism-channels@^0.8.0` only if the host wants a messaging ingress. Omitted, 0.7.0 hosts are unchanged.
|
|
76
|
+
- Host `authorize` stays deny-by-default. Group/topic traffic requires an explicit grant.
|
|
77
|
+
|
|
78
|
+
### 7. Connected apps and work HTTP
|
|
79
|
+
|
|
80
|
+
Identity-bound MCP connected-app sessions admit host-selected transports and register prefixed tools. Google Workspace and Microsoft 365 HTTP adapters live under `@arnilo/prism-work/connectors`. Slack MCP wrap and Open Connector sidecar remain examples, not core. See [connected apps](connected-apps.md) and [work connectors](work-connectors.md).
|
|
81
|
+
|
|
82
|
+
#### Migration Actions
|
|
83
|
+
- Wire `connected-apps` only with a host allow-list. Do not add Open Connector / Klavis / Nango as Prism dependencies.
|
|
84
|
+
|
|
85
|
+
### 8. Durable long runs
|
|
86
|
+
|
|
87
|
+
`AgentRunStateOptions.checkpointPolicy: "every-turn"` checkpoints at the provider-turn boundary. Host-only `decision: "continue"` resumes a crashed worker (never from AG-UI or the server boundary; rejected while an approval or ready tool call is pending). `RunOptions.turnPolicy` stops at a turn boundary with `stopReason: "host_policy"`. `snapshotRunBundle` returns a frozen redacted digest with zero store or network reads. `createClaimGroundingGuardrail` (stage `"output"`) blocks or flags numeric claims that no tool result or host evidence supports. `ErrorInfo.failureClass` types provider failures; `ModelCapabilities.toolCallStrictness` is advisory. See [durable runs](durable-runs.md), [run bundle](run-bundle.md), [guardrails](guardrails.md).
|
|
88
|
+
|
|
89
|
+
#### Migration Actions
|
|
90
|
+
- Omit `checkpointPolicy` / `turnPolicy` / the claim-grounding guardrail to keep 0.7.0 run behavior.
|
|
91
|
+
- `"continue"` is a host decision, not a client action.
|
|
92
|
+
|
|
93
|
+
### 9. Work sandbox and vendored skills
|
|
94
|
+
|
|
95
|
+
`@arnilo/prism-work/sandbox` plus `createWorkComposition` run office/exec in an injected Docker sandbox; connectors stay on the host. The package ships `docx`, `xlsx`, `powerpoint`, `pdf` skills. See [work sandbox](work-sandbox.md) and [context and skills](context-and-skills.md).
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## Operator / release honesty (not a host API break)
|
|
100
|
+
|
|
101
|
+
- `npm run test:postgres` writes gitignored `scripts/postgres-evidence.json` bound to `git rev-parse HEAD`. `release:gate` reports the Postgres surface as pass only when that evidence matches this tree. A stale phase baseline is **blocked**.
|
|
102
|
+
- Coverage artifact keys must equal live workspace package names (`@arnilo/prism-work`, not `@arnilo/prism-office`).
|
|
103
|
+
|
|
104
|
+
## Upgrade steps
|
|
105
|
+
|
|
106
|
+
1. Bump every `@arnilo/*` dependency and peer to `^0.8.0` (all **eleven** manifests cut together; a range that only *satisfies* 0.8.0 is refused by the release gate). The published predecessor is 0.7.0.
|
|
107
|
+
2. If the host imported `@arnilo/prism-office` or `integrations/work` / coding-tools `document-reader`, apply §1 before building.
|
|
108
|
+
3. Adopt §6–§9 only where the host wants channels, connected apps, durable-run checkpoints, or the work sandbox. Omitted, request bytes and tool lists stay 0.7.0.
|
|
109
|
+
4. Re-read §2–§5 if the host runs observational-memory workers, messaging channels, AG-UI, or inspects checkpoint ownership errors.
|
|
110
|
+
5. Build and run the host suite. No new session-store schema version ships in 0.8.0; channel journals and work HTTP state are new stores a 0.7.0 host never opened.
|
|
111
|
+
6. Optional: `PRISM_TEST_POSTGRES_URL=… npm run test:postgres` then `npm run release:gate` to reproduce this-tree Postgres evidence.
|
|
112
|
+
|
|
113
|
+
## Rollback
|
|
114
|
+
|
|
115
|
+
Pin the previous published line: `@arnilo/prism@0.7.0` and its siblings, exact pins per package. A 0.7.0 host does not load `@arnilo/prism-channels` or `@arnilo/prism-work`. Channel journal rows and work-package files written under 0.8.0 are invisible to 0.7.0, not rewritten. Session/checkpoint schema is unchanged across 0.7.0 → 0.8.0, so a pin rollback is store-safe for those adapters. Restore `@arnilo/prism-office` only from a 0.7.0 install — that package name is gone on 0.8.0.
|
|
116
|
+
|
|
117
|
+
Back up channel journals and work-sandbox volumes before a rollback if those 0.8.0 stores hold data you intend to keep.
|
|
118
|
+
|
|
119
|
+
## Related APIs
|
|
120
|
+
|
|
121
|
+
- [Migration guide](migration.md): the era index of migration cuts with replacement tables and rollback notes.
|
|
122
|
+
- [Migrate Prism 0.6 to 0.7](migrate-to-0.7.md): ACP MCP allow-list, model-router facade refusals, host-completeness additions.
|
|
123
|
+
- [Release and install](release-and-install.md): packed surfaces, install rules, support matrix, and the offline test budget.
|
|
124
|
+
- [Messaging channels](messaging-channels.md), [Connected apps](connected-apps.md), [Work tools](work-tools.md), [Durable runs](durable-runs.md): owning pages for the 0.8.0 additions.
|
|
@@ -0,0 +1,210 @@
|
|
|
1
|
+
# Migrate Prism 0.8 to 0.9
|
|
2
|
+
|
|
3
|
+
> **Status: 0.9.0** (attention budget axes, turn traces, cache-stable disclosure, per-turn tool narrowing, guardrail packs, background agents, checkpoint metadata, session search, deterministic turns, shared work scopes).
|
|
4
|
+
|
|
5
|
+
This document details migration steps, behavioral changes, and compatibility notes for upgrading from Prism 0.8.0 to 0.9.0.
|
|
6
|
+
|
|
7
|
+
0.9.0 is a lockstep minor for all **eleven** publishable packages. Node `>=22` stays the floor. **Nothing was removed**: no import path moved, no export was dropped, and every new surface defaults to 0.8 behavior — a host that only moves its dependency ranges keeps 0.8 request bytes, stores, and tool lists. The four deltas below sit inside existing surfaces, so they are readable without opting into anything.
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## Behavior changes inside existing surfaces
|
|
12
|
+
|
|
13
|
+
### 1. A limit death delivers three records, and only the last one is terminal
|
|
14
|
+
|
|
15
|
+
`run_limit_exceeded` was treated as terminal by the in-memory, NATS, and Postgres event sources and by AG-UI replay, so a consumer that stopped at the first breach record ended one record early — before the `budget_exhausted` attribution and before the run's terminal `error`. The terminal set is now exactly `agent_finished`, `agent_denied`, and `error`, decided by one exported predicate that every stream-ending site shares.
|
|
16
|
+
|
|
17
|
+
```ts
|
|
18
|
+
// before — the stream could end on the breach record
|
|
19
|
+
for await (const item of source.subscribe({ ... })) {
|
|
20
|
+
if (item.record.type === "run_limit_exceeded") break; // missed budget_exhausted and the error
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
// after — the breach and its attribution are not terminal; the error is
|
|
24
|
+
for await (const item of source.subscribe({ ... })) {
|
|
25
|
+
if (isTerminalAgentEventType(item.record.type)) break; // ends on the run's error
|
|
26
|
+
}
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
**Migration actions**
|
|
30
|
+
- Keep reading past `run_limit_exceeded` and `budget_exhausted`; the stream ends on `error`. A consumer that wants the attribution reads until `isTerminalAgentEventType(type)` is `true` (or the iterator ends).
|
|
31
|
+
- No configuration, no flag: this is the shipped delivery contract for pages, subscriptions, and replays. See [Agent events § Durable AgentEventSource](agent-events.md#durable-agenteventsource).
|
|
32
|
+
|
|
33
|
+
### 2. `provider_turn_finished` carries stop reason, budgets, tools, and cache metrics
|
|
34
|
+
|
|
35
|
+
The turn event gains attributed metadata: `stopReason` from one closed taxonomy (`end_turn`, `tool_calls`, `max_output_tokens`, `content_filter`, `abort`, `provider_error`, `unknown`), a `budgets` snapshot (`inputTokens?`, `inputCap?`, `runInputBudget?`, `runInputUsed`, `turns`, `maxTurns`), the effective tool menu as counts plus `tools.idsHash`, and provider-reported `cache` counts (`cacheReadTokens?`, `cacheWriteTokens?`, `hitRate?`). `agent_finished` carries the run-level `finishReason`/`stopDetail`, `AgentRunResult.stopReason` names host-policy and loop-ceiling stops, and the execution timeline adds `turns[i].stopReason` plus `timeline.exhaustion`.
|
|
36
|
+
|
|
37
|
+
```ts
|
|
38
|
+
source.subscribe({ ... }); // each provider_turn_finished.metadata:
|
|
39
|
+
// { latencyMs, stopReason: "tool_calls", budgets: { runInputUsed: 43_000, turns: 3, maxTurns: 16 }, tools: { count: 7, idsHash: "sha256:…" } }
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
**Migration actions**
|
|
43
|
+
- Consumers that deep-equal `metadata` (or reject unknown keys) must allow the new fields; consumers that read specific keys are unaffected.
|
|
44
|
+
- Read `metadata.cache` only when present — unknown cache usage stays absent rather than zero-filled.
|
|
45
|
+
|
|
46
|
+
### 3. Progressive disclosure is cache-stable
|
|
47
|
+
|
|
48
|
+
Late-expanding context (skill bodies, deferred tool schemas, loaded references) now lands at cache-stable positions: the request tail, or an explicit documented invalidation of the segment that changed. The default group order and the catalogs' slot are unchanged, so a host that never loads late context sends the same bytes as 0.8; hosts that do get append-only growth instead of a rewritten prefix.
|
|
49
|
+
|
|
50
|
+
```ts
|
|
51
|
+
// assert it against your own assembly (fixture provider, network-free, no keys)
|
|
52
|
+
await runPrefixStabilityConformance({ agent, minContinuity: 0.95 }); // ≥95% shared serialized prefix per turn
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Sizing: measured 100% / 95.7% / 95.8% shared prefix on the padded fixture for three consecutive requests; `minContinuity` defaults to `0.95` and is checked over messages **and** tool schemas. Cache reads/writes and per-turn hit rate are now recorded on usage records and `provider_turn_finished.metadata.cache`. See [Prefix stability conformance](prefix-stability-conformance.md) and [Provider caching](provider-caching.md).
|
|
56
|
+
|
|
57
|
+
### 4. A provider that reports no usage is charged a labeled estimate
|
|
58
|
+
|
|
59
|
+
A usage-less provider used to contribute zero tokens. `AgentConfig.usageEstimation` now defaults to `"fallback"`: one labeled `TokenEstimate` is recorded at the existing usage seam, and the label survives everywhere the number goes.
|
|
60
|
+
|
|
61
|
+
```ts
|
|
62
|
+
const meter = session.contextMeter();
|
|
63
|
+
// { inputTokens: 43_000, source: "estimated", inputCap: 200_000, runInputBudget: 500_000, usedRatio: 0.215 }
|
|
64
|
+
const estimate = estimateMessageTokens(messages, "claude-sonnet-4.5"); // { tokens, confidence: "medium" | "low", … }
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
**Migration actions**
|
|
68
|
+
- Billing or reporting code must read the `estimated` flag (and `confidence`) rather than treating every usage row as provider truth; reported usage always wins and is never overwritten.
|
|
69
|
+
- Set `usageEstimation: "off"` to keep the 0.8 zero-for-no-usage behavior. Estimates charge the token counters for usage-less vendors but never a price, so a configured `maxCost` stays fail-closed.
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## Additive surfaces (inert unless wired)
|
|
74
|
+
|
|
75
|
+
### 5. Attention budget axes and durable folding
|
|
76
|
+
|
|
77
|
+
`attentionCompiler.trigger` replaces the single `triggerRatio` gate with one axis, a predicate, or an any-of array: `{ kind: "input_ratio", ratio }` (the legacy axis), `{ kind: "run_input_ratio", ratio }` (fires against `RunLimits.maxInputTokens` — the case that used to be inert when the run cap sat below the model window), `{ kind: "token_floor", tokens }`, and a predicate function. Omitted, `triggerRatio` (default `0.75`) is the only axis and behavior is byte-identical to 0.8.
|
|
78
|
+
|
|
79
|
+
```ts
|
|
80
|
+
const attention = createAttentionCompiler(
|
|
81
|
+
{ trigger: [{ kind: "run_input_ratio", ratio: 0.75 }, { kind: "token_floor", tokens: 120_000 }], durable: true },
|
|
82
|
+
{ model, runInputBudget: limits.maxInputTokens },
|
|
83
|
+
);
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Sizing: one session-store write per fold (not per turn); folding stays default-off, and `durable: true` requires a checkpoint store (`runState`) or the run throws `AgentRunStateError` before its first provider turn. See [Attention compiler](attention-compiler.md).
|
|
87
|
+
|
|
88
|
+
### 6. Per-turn tool narrowing
|
|
89
|
+
|
|
90
|
+
`AgentConfig.toolNarrowing` / `RunOptions.toolNarrowing` (run wins) is a host callback invoked before each provider turn: it receives `{ turn, lastAssistantText?, toolIds }` and must return a subset of the run grant. Extra or unknown names are dropped — the runtime emits `tool_narrowing_clamped` with the dropped names — and a throw fails the turn instead of sending a partial schema.
|
|
91
|
+
|
|
92
|
+
```ts
|
|
93
|
+
await session.run("fix the failing test", {
|
|
94
|
+
toolNarrowing: async ({ turn, toolIds }) => (turn > 2 ? toolIds.filter((id) => id === "read" || id === "edit") : toolIds),
|
|
95
|
+
});
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Sizing and cache cost: changing the toolset rewrites provider schemas, so pair narrowing with tool search or deferred disclosure (where the tail contract above applies), and read menu identity from `provider_turn_finished.metadata.tools.idsHash` — identical consecutive subsets keep the same hash. Absent callback: 0.8 menu, byte-identical. See [Tools](tools.md).
|
|
99
|
+
|
|
100
|
+
### 7. Guardrail packs
|
|
101
|
+
|
|
102
|
+
`AgentSessionConfig.guardrailPacks` (or `compileGuardrailPacks(refs)` for hosts that dispatch tools directly) compiles declarative, restrictive-only rule sets onto the tool stages once per session. Four built-ins ship: `coding-standard` (`no-unrelated-file-edits`, `no-test-rewrites`), `destructive-commands`, `validation-respect`, and `secrets-hygiene`. Every pack ships a trajectory scorer (`createGuardrailPackScorer`) so enforcement can be graded, and pack denials are attributed as `pack:<pack>/<rule>`.
|
|
103
|
+
|
|
104
|
+
```ts
|
|
105
|
+
const agent = createAgent({ /* … */, session: { guardrailPacks: ["secrets-hygiene", "destructive-commands"] } });
|
|
106
|
+
// compiled from existing seams: interruptBeforeTool, the extension kernel, enforceExecutionPolicy
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
Sizing: `guardrailPacks` accepts at most 8 packs and 64 rules per pack; containment in `coding-standard` is lexical (`options.roots` defaults to `[process.cwd()]`, symlinks are not resolved), so an `ExecutionPolicy` stays the hard boundary. See [Guardrails](guardrails.md#guardrail-packs).
|
|
110
|
+
|
|
111
|
+
### 8. Background (session-lifetime) child agents and child-event passthrough
|
|
112
|
+
|
|
113
|
+
`delegate` / `delegateAsync` / `spawn_agent` accept `lifetime: "session"`, `report: "on-complete" | "milestones" | "stream"`, `milestone`, and `budgetShare`; a host `SupervisorChild.policy` sets the ceiling and a model request can only narrow it (report is clamped, `everyTurns` can only be raised, share takes the lower value, session lifetime must be host-enabled). Session-lifetime children survive caller turns until `cancel_agent` / `cancel(delegationId)`. New events: `child_milestone`, `child_failed` (with the plan-087 `RunLimitBreach` attribution), `delegation_child_events_capped`, `delegation_child_events_coalesced`.
|
|
114
|
+
|
|
115
|
+
```ts
|
|
116
|
+
const { delegationId } = await supervisor.delegateAsync({ childId: "researcher", input: "survey the repo", lifetime: "session", report: "milestones", milestone: { everyTurns: 3 }, budgetShare: 0.25 });
|
|
117
|
+
supervisor.subscribe(); // … child_milestone … child_failed (on a limit or error)
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Sizing: child events per delegation 256/4096, child-event bytes 32 KiB/256 KiB, child events per second 10/1000 (default/hard, per child) — 10/s is trivial for a UI, raise it only for a child whose tool events are the UI. Exceeding rate coalesces into one `delegation_child_events_coalesced` marker with the dropped count (never throws). Defaults are exactly 0.8: `lifetime: "task"`, `report: "on-complete"`, no milestone, no share, no subscription. See [Supervisors](supervisors.md) and [Multi-agent patterns](multi-agent-patterns.md).
|
|
121
|
+
|
|
122
|
+
### 9. Checkpoint sidecar metadata and cross-layer restore hooks
|
|
123
|
+
|
|
124
|
+
Hosts attach an opaque, redacted metadata map (≤4 KiB) to every checkpoint record — git commit, document version, workspace fingerprint — without charging `maxStateBytes`, and register restore hooks that put each recorded layer back before a resume claims the run.
|
|
125
|
+
|
|
126
|
+
```ts
|
|
127
|
+
createAgentRunLifecycle({ runState: { checkpointMetadata: () => ({ gitCommit: head, docVersion: "v12" }) },
|
|
128
|
+
restoreHooks: [async ({ metadata, signal }) => { await checkout(metadata.gitCommit, { signal }); }] });
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
Sizing: `MAX_AGENT_RUN_METADATA_BYTES` is 4 KiB (fixed, no override), the whole map is redacted unconditionally (no public-key exemption), and each hook has a 10-second default timeout (`DEFAULT_CHECKPOINT_RESTORE_TIMEOUT_MS`) with sequential execution; any hook failure aborts the restore with `{ hook, error }`. Legacy records without metadata read as `undefined`, and an oversize or non-string map reads as absent rather than failing a resume — unused, behavior is unchanged. See [Durable runs](durable-runs.md).
|
|
132
|
+
|
|
133
|
+
### 10. Bounded workspace session search
|
|
134
|
+
|
|
135
|
+
`SessionStore.searchSessions?(query)` is part of the store contract: filters by workspace root (`metadata.workspaceRoot`), time, provider/model, label/summary, entry kind, ownership, and an optional full-text `query`; hits carry `sessionId`, optional `leafId`, and the matched entry pointer (`entryId`, `runId`, 1-based `turn`, store `score`, bounded `snippet`) — never credentials or whole transcripts. SQLite FTS5 and the Postgres `tsvector` column are maintained additively at append time (migration 004, no background job); memory and JSONL stores scan linearly through the shared `searchLinearSessions` matcher.
|
|
136
|
+
|
|
137
|
+
```ts
|
|
138
|
+
const page = await store.searchSessions!({ workspaceRoot: "/repo", query: "flake", kind: "any", limit: 20 });
|
|
139
|
+
const { page } = await searchSessions({ bySession, leafBySession, query: "flake" }); // linear caps apply
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Sizing: on the 100k-turn fixture the index is 18.8% of transcript page bytes (stored tool output is never indexed) and query p95 is 38 ms against the 100 ms ceiling; unindexed stores are O(corpus) per query and accept `maxLinearSessions` / `maxLinearEntries` / `maxLinearBytes` overrides bounded by their hard caps. See [Session stores](session-stores.md) and `examples/session-search.ts`.
|
|
143
|
+
|
|
144
|
+
### 11. Deterministic no-model turns
|
|
145
|
+
|
|
146
|
+
The `beforeProviderTurn` middleware hook receives `BeforeProviderTurnPayload` (`sessionId`, `runId`, `turn`, `userText`) and may answer the turn from host data by returning a `DeterministicTurnAnswer` — no provider request, zero model cost, no hallucination surface. Answers are validated (`validateDeterministicTurnAnswer` / `resolveDeterministicTurn`), the turn is recorded as `deterministic` on the timeline and in usage, and `DeterministicTurnProvenance` names the middleware that produced it. `createDeterministicTurnScorer` grades the behavior in evals.
|
|
147
|
+
|
|
148
|
+
```ts
|
|
149
|
+
middleware.use<BeforeProviderTurnPayload>("beforeProviderTurn", async (payload, next) =>
|
|
150
|
+
payload.userText.startsWith("status:")
|
|
151
|
+
? { ...payload, answer: { text: await hostStatus(payload.userText), provenance: { middleware: "status" } } }
|
|
152
|
+
: next(payload));
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
Sizing: no provider turn, no usage beyond a zero-cost record; the hook runs only for turns that reach the provider boundary (a turn already ended by a run limit, host turn policy, or suspension never reaches it), and host middleware is trusted code — it must not use the hook to bypass `RunLimits` or guardrails. See [Middleware hooks](middleware-hooks.md#no-model-turns-beforeproviderturn).
|
|
156
|
+
|
|
157
|
+
### 12. Shared work scopes for observational memory
|
|
158
|
+
|
|
159
|
+
`om.attach(session, { sharedScopes })` lets several sessions contribute to and read one observational-memory scope under explicit owner grants (`controller.grant(scopeId, principalIds)` / `revoke`). Only ids bound to that exact scope are shared; the owner branch is the only grant authority; every resolve re-reads it, so revocation lands on the next read, and `onScopeAccess` audits each grant/denial.
|
|
160
|
+
|
|
161
|
+
```ts
|
|
162
|
+
om.attach(session, {
|
|
163
|
+
appendEntry: (entry, options) => store.append(entry, options),
|
|
164
|
+
sharedScopes: { "build-42": { ownerSessionId, entries: (id) => store.list(id) } },
|
|
165
|
+
onScopeAccess: (event) => audit.info("om.scope.access", event),
|
|
166
|
+
});
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
Sizing: one local write per append (flush stays local — no second writer on a branch); one branch read plus fold per participating branch per context resolve and per shared-scope recall, not per observation; 1,024 principals per scope and 256-character principal ids, on top of the existing scope caps (256 scopes, depth 8, 4,096 binds, 512-character labels). Session-private scopes stay the default: with no `sharedScopes` configured, behavior is byte-identical to 0.8. See [Compaction and observational memory](compaction-observational-memory.md#shared-work-scopes-opt-in) and `examples/shared-work-scope.ts`.
|
|
170
|
+
|
|
171
|
+
### 13. Retrieval revocation and a zero-service reranker
|
|
172
|
+
|
|
173
|
+
Deletion and revocation propagate through derived artifacts: `createDeletionPropagator` deletes vector rows and then hands the invalidation set (`collectInvalidationIds` / `listInvalidatedIds`) to host handlers, and `repointSource` / `retireWikiSources` (plus the `createWikiDeletionHandler` / `createWikiRepointHandler` helpers) keep wiki pages and summaries consistent. `createAccessRecheck` rechecks governed sources per query and reports denials through an audit sink. Reranking no longer needs a research project: `resolveReranker({ kind: "local" })` / `createLocalReranker()` runs an in-process cross-encoder behind the `LocalRerankRuntime` seam, with the same `runRerankerConformance` contract as every other reranker; TEI and hosted adapters are unchanged.
|
|
174
|
+
|
|
175
|
+
```ts
|
|
176
|
+
const reranker = resolveReranker({ kind: "local" }); // Xenova/bge-reranker-base via a host-owned runtime
|
|
177
|
+
const access = createAccessRecheck({ store, onDenied: audit.warn });
|
|
178
|
+
const propagator = createDeletionPropagator({ store, handlers: [createRagDeletionHandler(vectors), createWikiDeletionHandler(wiki)] });
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
Sizing: the local reranker declares no inference dependency — the built-in loader resolves `@huggingface/transformers` at first use (pass `runtime` to inject your own, or `allowRemoteModels: false` for a no-network posture after the model is cached); propagation and repoint walks are bounded by `HARD_PROPAGATION_EDGES` / `HARD_REPOINT_RECORDS` and fail closed past them. See [RAG](rag.md#local-reranker), [Embeddings](embeddings.md), and [Knowledge sync](knowledge-sync.md).
|
|
182
|
+
|
|
183
|
+
---
|
|
184
|
+
|
|
185
|
+
## Operator / release honesty (not a host API break)
|
|
186
|
+
|
|
187
|
+
- **Compatibility baseline regenerated**: `+119` public names across `@arnilo/prism` (+49), `@arnilo/prism-memory` (+60), and `@arnilo/prism-core` (+10); **zero removals** and zero renames. One declaration change is consumer-visible at the type level: the `recordUsage` callback accepted by `generateProviderTurn` / `generateWithRetry` now returns `Promise<Usage | undefined>` instead of `Promise<void>`, so a hand-written callback that returned nothing must return the usage (or `undefined`).
|
|
188
|
+
- **Budgets rebaselined with recorded reasons**: root packed/unpacked/file count, per-package export ceilings, and the non-null assertion ratchet carry the measured 0.9.0 values and the plans that moved them.
|
|
189
|
+
- **This-tree Postgres evidence**: `release:gate` reports the durable Postgres surface as pass only when `scripts/postgres-evidence.json` matches the current `git rev-parse HEAD`; a stale phase baseline is blocked rather than inherited.
|
|
190
|
+
- **Version literals agree** across all eleven manifests, the lockfile, `src/index.ts`, the docs banner, the release workflow tag lists, and the generated package-truth artifact (`scripts/version-literal-gate.test.mjs`).
|
|
191
|
+
|
|
192
|
+
## Upgrade steps
|
|
193
|
+
|
|
194
|
+
1. Bump every `@arnilo/*` dependency and peer to `^0.9.0` (all **eleven** manifests cut together; a range that only *satisfies* 0.9.0 is refused by the release gate). The published predecessor is 0.8.0.
|
|
195
|
+
2. Build and run the host suite. No import path moved, so compile errors should be limited to the `recordUsage` callback return type above and to code that deep-equals `provider_turn_finished.metadata`.
|
|
196
|
+
3. Re-read §1–§4 if the host tails durable agent events, parses provider-turn metadata, uses progressive disclosure or prompt caching, or bills usage for vendors that report no usage.
|
|
197
|
+
4. Adopt §5–§13 only where the host wants the new surfaces. Omitted, request bytes, stores, and tool lists stay 0.8.
|
|
198
|
+
5. Run the new migration 004 on SQLite/Postgres stores if the host wants indexed session search; existing tables and columns are untouched, and 0.8 stores open unchanged.
|
|
199
|
+
6. Optional: `PRISM_TEST_POSTGRES_URL=… npm run test:postgres` then `npm run release:gate` to reproduce this-tree Postgres evidence.
|
|
200
|
+
|
|
201
|
+
## Rollback
|
|
202
|
+
|
|
203
|
+
Pin the previous published line: `@arnilo/prism@0.8.0` and its siblings, exact pins per package. Session, checkpoint, and ledger schema are unchanged across 0.8.0 → 0.9.0 apart from the additive session-search index (migration 004), which a 0.8.0 process never reads; observability rows written under 0.9.0 carry extra metadata fields that 0.8.0 ignores. Revert host config to 0.8.0 semantics by dropping `attentionCompiler.trigger` / `durable`, `toolNarrowing`, `guardrailPacks`, `usageEstimation`, `checkpointMetadata` / `restoreHooks`, `sharedScopes`, and the session-lifetime child options — every default already matches 0.8.0.
|
|
204
|
+
|
|
205
|
+
## Related APIs
|
|
206
|
+
|
|
207
|
+
- [Migration guide](migration.md): the era index of migration cuts with replacement tables and rollback notes.
|
|
208
|
+
- [Migrate Prism 0.7 to 0.8](migrate-to-0.8.md): work-family import map, messaging channels, connected apps, durable runs.
|
|
209
|
+
- [Release and install](release-and-install.md): packed surfaces, install rules, support matrix, and the offline test budget.
|
|
210
|
+
- [Agent events](agent-events.md), [Runs and usage](runs-and-usage.md), [Observability](observability.md), [Tools](tools.md), [Guardrails](guardrails.md), [Supervisors](supervisors.md), [Session stores](session-stores.md): owning pages for the 0.9.0 additions.
|